From nobody Tue Feb 10 23:53:30 2026 Received: from mail-oo1-f46.google.com (mail-oo1-f46.google.com [209.85.161.46]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 1698A14EC7F for ; Thu, 16 May 2024 15:49:39 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.161.46 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1715874581; cv=none; b=XTw752Q5MOOFMcH0P2DO5BH0YAOLx6/Bz7N3Uq3M1wmiUHETq8bT6FwTSPkUjuSrvOBIdaMURzbOuoe0RYa6d8HW8c9Bnbm1MKTIqRKE3YHkyElZoC9cxb4C6v2akOYoiiW9e5fvtF9RM3p5xjrIdSws2I8Bb9jnIcUfMaJwmbQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1715874581; c=relaxed/simple; bh=JaUqxQHgv5XV0nLiCJPy0w1saO04+8/J9qQEQZEGkUI=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=cHQLAQEDHIxSDFgYfMO1yAuqAYQiPZnkZzVTLaWQyK/udcM0jtclWchfcTw+aiNcduw12AU3en/T8gntnqhv2CThEbPH40OspmkVEI9Szh8R4iuxvFO+9BltFzBjI043JLqCBCF8YMVsALCQ61iUMMdy2YxoFNMrSKwo/ObTIgM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=iMcNt0zj; arc=none smtp.client-ip=209.85.161.46 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="iMcNt0zj" Received: by mail-oo1-f46.google.com with SMTP id 006d021491bc7-5b21e393debso248444eaf.2 for ; Thu, 16 May 2024 08:49:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1715874579; x=1716479379; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=hOy8P+m9e9TVgnsDHnWWNjFRdWyhKRoqoSc5hCvwSV4=; b=iMcNt0zjBTcw9K8p/AxkihPgFexQ+JjZ6/JPWmTrLx9CPwJRrr4gPBLQ0qbTIpgmTf pmohl8HNLajApgnpyjn9BG8ZlnA5N8UtzrJSKU/y7V+QbFNSLCaNiiyRrKaQ3xhJSQkk WRgKAJNx0vjG7cvQ3RfxtZ/99SPhRbwgzetce6/NZdhv3aNYRp/b9lsZQ3G70d8rwU72 3w/qfz+B2qpKS6OcxZyAchHmcZGca231YK/GvSFAoLdYnPNiYjbahn5Rkx7vcFKaf53e 4zZTAu2t/08vRGwYbFyxjCUAE/NWoG6YOlj7vza55rnNjbMHCNCpKJT1A+rfaO0bvxHn FNeg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1715874579; x=1716479379; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=hOy8P+m9e9TVgnsDHnWWNjFRdWyhKRoqoSc5hCvwSV4=; b=RHYPstuXf0k7yhK+E2AHoywBI+GGB9HJmXUNIR2KP41wEiA54Kw4OOqPhaaqebFB6l X5hvzHCTZh5AjO4MS+j2lZUjfzyMRIoxLP4IyIH5EHiWZvTHIrr/HnMT1EQ4om88yiyy TrthQD62kn/RPXffG79CMY/6PcC1lHqsehDDdTqSh4JorqikKtZ8v4eD8BDGxZXQ0+12 Ksthuzs7EcuPa2z40igZVpHeKH2jY/Kbpef09oK+P2SflRAEv15rGJAJD14RJRhoAb28 4SMH8z7Wr9yFykkxpUF7PVtq1PCrax+pneNRoyB3R6FtCepfwSMpaqZK9RkhY5flpLcF TGmA== X-Gm-Message-State: AOJu0Yzi9auJWqmOjzPLAMPst6x6FrNZPPa3VCroQ7ZjUaxuftBVDXUR Rq8bqDuzyVvR/NcWEncLDpjZA5xr5pROPbmAXv7qAVfCOCJ+Ppm76PhRBF0k X-Google-Smtp-Source: AGHT+IFe4JxeqXFfEEedGDdC/7vfVPMAfs7lQs5p32UhZ+Gvqi7D8XSTAiVSEh6jnICim3SD5rMfrw== X-Received: by 2002:a05:6870:7b52:b0:239:701f:fc32 with SMTP id 586e51a60fabf-24172a23544mr21892680fac.9.1715874578667; Thu, 16 May 2024 08:49:38 -0700 (PDT) Received: from localhost.localdomain ([143.166.81.254]) by smtp.gmail.com with ESMTPSA id 586e51a60fabf-2412a3c82edsm3562563fac.12.2024.05.16.08.49.37 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 16 May 2024 08:49:38 -0700 (PDT) From: Stuart Hayes To: linux-kernel@vger.kernel.org, Greg Kroah-Hartman , "Rafael J . Wysocki" , Tanjore Suresh , Martin Belanger , Oliver O'Halloran , Daniel Wagner , Keith Busch , Lukas Wunner , David Jeffery , Jeremy Allison , Jens Axboe , Christoph Hellwig , Sagi Grimberg , linux-nvme@lists.infradead.org Cc: Stuart Hayes Subject: [PATCH v6 1/4] driver core: don't always lock parent in shutdown Date: Thu, 16 May 2024 10:49:17 -0500 Message-Id: <20240516154920.221445-2-stuart.w.hayes@gmail.com> X-Mailer: git-send-email 2.39.3 In-Reply-To: <20240516154920.221445-1-stuart.w.hayes@gmail.com> References: <20240516154920.221445-1-stuart.w.hayes@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Don't lock a parent device unless it is needed in device_shutdown. This is in preparation for making device shutdown asynchronous, when it will be needed to allow children of a common parent to shut down simultaneously. Signed-off-by: Stuart Hayes Signed-off-by: David Jeffery --- drivers/base/core.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/drivers/base/core.c b/drivers/base/core.c index 5f4e03336e68..e63177314e86 100644 --- a/drivers/base/core.c +++ b/drivers/base/core.c @@ -4856,7 +4856,7 @@ void device_shutdown(void) spin_unlock(&devices_kset->list_lock); =20 /* hold lock to avoid race with probe/release */ - if (parent) + if (parent && dev->bus && dev->bus->need_parent_lock) device_lock(parent); device_lock(dev); =20 @@ -4880,7 +4880,7 @@ void device_shutdown(void) } =20 device_unlock(dev); - if (parent) + if (parent && dev->bus && dev->bus->need_parent_lock) device_unlock(parent); =20 put_device(dev); --=20 2.39.3 From nobody Tue Feb 10 23:53:30 2026 Received: from mail-ot1-f48.google.com (mail-ot1-f48.google.com [209.85.210.48]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 78F7114F114 for ; Thu, 16 May 2024 15:49:42 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.48 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1715874584; cv=none; b=Rc8KGGVCxGf7WUbLCGcHHQMXbJ4jeN/dsCwO8BSw5mcJOi1y2Ojb9VRtycgtQh4McGkcym8exa15zSYFLUfNYUmXa3p33GWdS/qhnw5zTl2aZEuFb2u5keiZXX3/PWa0Np4cI6pNM0pxALX9VtTecywrHlaJtuUGxo18Qj9XT7g= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1715874584; c=relaxed/simple; bh=EVxLaUmi4wnckNMNHXzYTMXdTPVbbQZdt0IOYxoXGtM=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=cYGjPeOQTSbfnKhKzWtCzuHZuOeNcZCl6wUFa530YHVRhB5H0ntFi3dC8Ss3q+NziQp3JPGGX5aCfbfo7ScaP5u18WzYSFGiPdjftvA6biEYYuG3ZnA1/lIKKW+JQ/O+TERh2sx2yqBhnkW8o80rPD0VhIQZaYOOA3Aoo3N+vVo= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=OCKtdax8; arc=none smtp.client-ip=209.85.210.48 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="OCKtdax8" Received: by mail-ot1-f48.google.com with SMTP id 46e09a7af769-6f054c567e2so216306a34.2 for ; Thu, 16 May 2024 08:49:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1715874581; x=1716479381; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=OdxvUmUr/n3drhohz+SzK2EeawufACujFjOU7myXx/g=; b=OCKtdax8RaEjcT6ll2wchFCdJgdxIlTMQLmuPBH1y3KX5y6VnqJ9UlGT8M6/cJyr+y 0vcDOAttls90GmtrQJuffSk9rt9wbJq3/KQRh7QOxPnwyE7TYNgIPDgDaSqWk03kZxg3 Wbz8ncPYd6NF/6yXAT6RgAPiqXMa97yKdK1VssZk17Bc9GsAFOIgeDw0qtctBuBtoAec UfoBI3a/9FxcgJ+fJMjTJZdleTpt6R6wMlu+0lY9i7SHgUEOLN+eS+/X0ozNzSTpBz2R /IWyiOhF+N4yflWIeow4xmKkz9CaTQ/QIfK8NRjF6Kay8isZe+JrcVjnJVd6cIf3zrAn jZAg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1715874581; x=1716479381; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=OdxvUmUr/n3drhohz+SzK2EeawufACujFjOU7myXx/g=; b=LUMJyHWc58m5IRaWiJKlO78ZMJGym9XEpmYANOHxfrp4fTrH/o+b4n1D+oMjRGnmq/ lOhR8K5ja3vkuQgiN80fobDM+c5qE8CyRPaF+zi6gIWOMs6XKpx4UxNw1lRyLRN9bImc skqrGItLpgmKNc91F8NoHm4yeuFpesKugYPmaToWMxrcHZmNqP+BZLzDAKpFwBC4LPwb JWTNq/wdQ3flx97Y5923hRUqe+oOlgIUeM8rcXd6UzpQqxA0OW8BT9uRrB9wJoVyCbUP O2GBooKYLPCwOGvtwFwLDAM78z9qWNINzkHXc8EQZewOkex4atfQDy57l2+zA8LizOuG oQjQ== X-Gm-Message-State: AOJu0YzFnQ8wynGhIxlxl/YH1mpGX7a/psX0RfJlyvnPcv2iwJpaIGXy ZFuApmA9B8/BsY2FQ9zmAOOtlcdYJYdQw2pLr/CrBL+S1u6lnX7QpPP24dIF X-Google-Smtp-Source: AGHT+IHTwAQvrUcBgReDLYo3j6uEenDDM0m6mgD/4TKTn44LluYWEepiGRIUA0sp3FRMMd0xbjb3LA== X-Received: by 2002:a05:6870:c45:b0:239:7802:cd5 with SMTP id 586e51a60fabf-24172a4205emr22868219fac.6.1715874580898; Thu, 16 May 2024 08:49:40 -0700 (PDT) Received: from localhost.localdomain ([143.166.81.254]) by smtp.gmail.com with ESMTPSA id 586e51a60fabf-2412a3c82edsm3562563fac.12.2024.05.16.08.49.39 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 16 May 2024 08:49:40 -0700 (PDT) From: Stuart Hayes To: linux-kernel@vger.kernel.org, Greg Kroah-Hartman , "Rafael J . Wysocki" , Tanjore Suresh , Martin Belanger , Oliver O'Halloran , Daniel Wagner , Keith Busch , Lukas Wunner , David Jeffery , Jeremy Allison , Jens Axboe , Christoph Hellwig , Sagi Grimberg , linux-nvme@lists.infradead.org Cc: Stuart Hayes Subject: [PATCH v6 2/4] driver core: separate function to shutdown one device Date: Thu, 16 May 2024 10:49:18 -0500 Message-Id: <20240516154920.221445-3-stuart.w.hayes@gmail.com> X-Mailer: git-send-email 2.39.3 In-Reply-To: <20240516154920.221445-1-stuart.w.hayes@gmail.com> References: <20240516154920.221445-1-stuart.w.hayes@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Make a separate function for the part of device_shutdown() that does the shutown for a single device. This is in preparation for making device shutdown asynchronous. Signed-off-by: Stuart Hayes Signed-off-by: David Jeffery --- drivers/base/core.c | 66 ++++++++++++++++++++++++--------------------- 1 file changed, 36 insertions(+), 30 deletions(-) diff --git a/drivers/base/core.c b/drivers/base/core.c index e63177314e86..e76cba51513a 100644 --- a/drivers/base/core.c +++ b/drivers/base/core.c @@ -4819,6 +4819,41 @@ int device_change_owner(struct device *dev, kuid_t k= uid, kgid_t kgid) } EXPORT_SYMBOL_GPL(device_change_owner); =20 +static void shutdown_one_device(struct device *dev) +{ + /* hold lock to avoid race with probe/release */ + if (dev->parent && dev->bus && dev->bus->need_parent_lock) + device_lock(dev->parent); + device_lock(dev); + + /* Don't allow any more runtime suspends */ + pm_runtime_get_noresume(dev); + pm_runtime_barrier(dev); + + if (dev->class && dev->class->shutdown_pre) { + if (initcall_debug) + dev_info(dev, "shutdown_pre\n"); + dev->class->shutdown_pre(dev); + } + if (dev->bus && dev->bus->shutdown) { + if (initcall_debug) + dev_info(dev, "shutdown\n"); + dev->bus->shutdown(dev); + } else if (dev->driver && dev->driver->shutdown) { + if (initcall_debug) + dev_info(dev, "shutdown\n"); + dev->driver->shutdown(dev); + } + + device_unlock(dev); + if (dev->parent && dev->bus && dev->bus->need_parent_lock) + device_unlock(dev->parent); + + put_device(dev); + if (dev->parent) + put_device(dev->parent); +} + /** * device_shutdown - call ->shutdown() on each device to shutdown. */ @@ -4855,36 +4890,7 @@ void device_shutdown(void) list_del_init(&dev->kobj.entry); spin_unlock(&devices_kset->list_lock); =20 - /* hold lock to avoid race with probe/release */ - if (parent && dev->bus && dev->bus->need_parent_lock) - device_lock(parent); - device_lock(dev); - - /* Don't allow any more runtime suspends */ - pm_runtime_get_noresume(dev); - pm_runtime_barrier(dev); - - if (dev->class && dev->class->shutdown_pre) { - if (initcall_debug) - dev_info(dev, "shutdown_pre\n"); - dev->class->shutdown_pre(dev); - } - if (dev->bus && dev->bus->shutdown) { - if (initcall_debug) - dev_info(dev, "shutdown\n"); - dev->bus->shutdown(dev); - } else if (dev->driver && dev->driver->shutdown) { - if (initcall_debug) - dev_info(dev, "shutdown\n"); - dev->driver->shutdown(dev); - } - - device_unlock(dev); - if (parent && dev->bus && dev->bus->need_parent_lock) - device_unlock(parent); - - put_device(dev); - put_device(parent); + shutdown_one_device(dev); =20 spin_lock(&devices_kset->list_lock); } --=20 2.39.3 From nobody Tue Feb 10 23:53:30 2026 Received: from mail-oo1-f45.google.com (mail-oo1-f45.google.com [209.85.161.45]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 71DE214F135 for ; Thu, 16 May 2024 15:49:44 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.161.45 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1715874586; cv=none; b=oqWMf2ZkjSXe61bwP9FPa4E5Nw8TviSPquwQrGhMLPVKNakUB59xcoinEsJ2KSGbJS+bJBb6ZzMWkT2mx7STXY4oCxknXlGMB+LlY2X/Bx1bA3zF9dBBMHaSorah4OpFIRbBatwqoBx2B+9naVVscrqwJdJPP+M16GhoW0TpXB4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1715874586; c=relaxed/simple; bh=LUF6oGuFMfH5spi+UBEY6opX99jQwTcQMHo+dM2Fp4A=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=DWLUmU3XkSOmpUjWhkPziVVFY6PBRBZBXK/lY7lklB5wqw5EfelVn4hbhj/LDT6cwLUhY3jb7pg7pD3lHF5xlNhZUup8HtYZu2XXSX7PF1K1R1LfxrNQ3L4V1EKuL+wkwI+mnh8xrpesUEd6m8K6Jt1DbuhFIcRxKnZ65yu3PU0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=YyWxF0l2; arc=none smtp.client-ip=209.85.161.45 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="YyWxF0l2" Received: by mail-oo1-f45.google.com with SMTP id 006d021491bc7-5b2a2ef4e4cso245539eaf.0 for ; Thu, 16 May 2024 08:49:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1715874583; x=1716479383; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=LXa9ugcDPjbQT7wpPd+qHkyY+/Gaic2gMm244nq+Jqk=; b=YyWxF0l2GzUHekPs1yZNnPINwPBFwx1zbrE6+U0KqZ6lJhHJl7DXRHWpikOEal/YCB rJh0wW69WhZIGPBpnQfMgafcARjoSw3yUwazR239LiaT61CX5zKDc0e1SIekcBWEItZG K6yFxSQYea3ykICJrPSxdh0DFejrjAm/xj0dLCJYLuY8+5iLtfYcQclMi20x24FnuZ4N vba+sXtPqITJHYrkanOspRKODCw0TdJ0sgHC40CUB5lwKLRHgjvrRdvWdyJEJN4GcUf5 Avpw25DTUeS06xiyOZH0+Eu2Df/zVmYgz/GJgob/ISf8zEuaOc0d/dm1ARNSGTJhci5O CUXw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1715874583; x=1716479383; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=LXa9ugcDPjbQT7wpPd+qHkyY+/Gaic2gMm244nq+Jqk=; b=WOERqtsbFjw/4oYwx0NXGMdMFcw5wqKB+34kp3H7rNzT6HOSTlZ+G7yABhBsqoZmvM 8Bbh9eWAIAPuh8qZtvSszoQ+caFm/PgIuz10pja/FN/H9bTfbvG5Ia7oQP0KWX4TrgGY 2tKs2/mUTUQxGb89IIt7g9bkJBP9+z1IHc11++VMPJw/PGSh12aIICZHD9shRwtAeyH4 Ud6GEwfDeSAXmqYqBP1V8jXRXaKpQ+NhMtgggsZTtriSlYM/FBFx8nEH3Vt3k2qRd0lG zb6/PQBH/2V56aHbd94Mw3VwxWmVCgw1eAxrMwfNhK765nEdP0LDhJu7JsNAv0uk31/Y ADUQ== X-Gm-Message-State: AOJu0Yyd6h/4KOOXkzCagu8fEwHbaDH0fETLByR/gxjm/ztaU6zUvZN1 0l745S0LGGXQGeoqdpFetjI43ip5+fdUEQVscbQXAQNLNRKkH/x/mNvPyxvU X-Google-Smtp-Source: AGHT+IFVLy3acBgdcHb2N5/+8Adq12aS9hZ4Qh1zyO1bzJItFeguxh3NJD6h5WRqOlNAT7dhreUpqA== X-Received: by 2002:a05:6870:c69d:b0:239:c163:a400 with SMTP id 586e51a60fabf-24172ba92c4mr22194657fac.29.1715874582873; Thu, 16 May 2024 08:49:42 -0700 (PDT) Received: from localhost.localdomain ([143.166.81.254]) by smtp.gmail.com with ESMTPSA id 586e51a60fabf-2412a3c82edsm3562563fac.12.2024.05.16.08.49.41 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 16 May 2024 08:49:42 -0700 (PDT) From: Stuart Hayes To: linux-kernel@vger.kernel.org, Greg Kroah-Hartman , "Rafael J . Wysocki" , Tanjore Suresh , Martin Belanger , Oliver O'Halloran , Daniel Wagner , Keith Busch , Lukas Wunner , David Jeffery , Jeremy Allison , Jens Axboe , Christoph Hellwig , Sagi Grimberg , linux-nvme@lists.infradead.org Cc: Stuart Hayes Subject: [PATCH v6 3/4] driver core: shut down devices asynchronously Date: Thu, 16 May 2024 10:49:19 -0500 Message-Id: <20240516154920.221445-4-stuart.w.hayes@gmail.com> X-Mailer: git-send-email 2.39.3 In-Reply-To: <20240516154920.221445-1-stuart.w.hayes@gmail.com> References: <20240516154920.221445-1-stuart.w.hayes@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Add code to shut down devices asynchronously, while ensuring that each device is shut down before its parents & suppliers, and allowing devices that share a driver to be shutdown one at a time if necessary. Add /sys/kernel/async_shutdown to allow user control of this feature: safe: shut down all devices synchronously, unless driver prefers async shutdown (driver opt-in) (default) on: shut down all devices asynchronously, unless disabled by the driver (driver opt-out) off: shut down all devices synchronously Add async_shutdown to struct device_driver, and expose it via sysfs. This will be used to view or change driver opt-in/opt-out of asynchronous shutdown, if it is globally enabled. async: driver opt-in to async device shutdown (devices will be shut down asynchronously if async_shutdown is "on" or "safe") sync: driver opt-out of async device shutdown (devices will always be shut down synchronously) default: devices will be shutdown asynchronously if async_shutdown is "on" This can dramatically reduce system shutdown/reboot time on systems that have multiple devices that take many seconds to shut down (like certain NVMe drives). On one system tested, the shutdown time went from 11 minutes without this patch to 55 seconds with the patch. Signed-off-by: Stuart Hayes Signed-off-by: David Jeffery --- drivers/base/base.h | 3 + drivers/base/bus.c | 47 +++++++++++++ drivers/base/core.c | 129 +++++++++++++++++++++++++++++++++- include/linux/device/driver.h | 8 +++ 4 files changed, 186 insertions(+), 1 deletion(-) diff --git a/drivers/base/base.h b/drivers/base/base.h index 0738ccad08b2..ab80a0721b2e 100644 --- a/drivers/base/base.h +++ b/drivers/base/base.h @@ -10,6 +10,7 @@ * shared outside of the drivers/base/ directory. * */ +#include #include =20 /** @@ -97,6 +98,7 @@ struct driver_private { * the device; typically because it depends on another driver getting * probed first. * @async_driver - pointer to device driver awaiting probe via async_probe + * @shutdown_after - used during async shutdown to ensure correct shutdown= ordering. * @device - pointer back to the struct device that this structure is * associated with. * @dead - This device is currently either in the process of or has been @@ -114,6 +116,7 @@ struct device_private { struct list_head deferred_probe; struct device_driver *async_driver; char *deferred_probe_reason; + async_cookie_t shutdown_after; struct device *device; u8 dead:1; }; diff --git a/drivers/base/bus.c b/drivers/base/bus.c index daee55c9b2d9..403eecab22a3 100644 --- a/drivers/base/bus.c +++ b/drivers/base/bus.c @@ -10,6 +10,7 @@ */ =20 #include +#include #include #include #include @@ -635,6 +636,46 @@ static ssize_t uevent_store(struct device_driver *drv,= const char *buf, } static DRIVER_ATTR_WO(uevent); =20 +static ssize_t async_shutdown_show(struct device_driver *drv, char *buf) +{ + char *output; + + switch (drv->shutdown_type) { + case SHUTDOWN_DEFAULT_STRATEGY: + output =3D "default"; + break; + case SHUTDOWN_PREFER_ASYNCHRONOUS: + output =3D "enabled"; + break; + case SHUTDOWN_FORCE_SYNCHRONOUS: + output =3D "disabled"; + break; + default: + output =3D "unknown"; + } + return sysfs_emit(buf, "%s\n", output); +} + +static ssize_t async_shutdown_store(struct device_driver *drv, const char = *buf, + size_t count) +{ + if (!capable(CAP_SYS_BOOT)) + return -EPERM; + + if (!strncmp(buf, "disabled", 8)) + drv->shutdown_type =3D SHUTDOWN_FORCE_SYNCHRONOUS; + else if (!strncmp(buf, "enabled", 2)) + drv->shutdown_type =3D SHUTDOWN_PREFER_ASYNCHRONOUS; + else if (!strncmp(buf, "default", 4)) + drv->shutdown_type =3D SHUTDOWN_DEFAULT_STRATEGY; + else + return -EINVAL; + + return count; +} + +static DRIVER_ATTR_RW(async_shutdown); + /** * bus_add_driver - Add a driver to the bus. * @drv: driver. @@ -697,6 +738,12 @@ int bus_add_driver(struct device_driver *drv) } } =20 + error =3D driver_create_file(drv, &driver_attr_async_shutdown); + if (error) { + pr_err("%s: async_shutdown attr (%s) failed\n", + __func__, drv->name); + } + return 0; =20 out_del_list: diff --git a/drivers/base/core.c b/drivers/base/core.c index e76cba51513a..1f71282741f8 100644 --- a/drivers/base/core.c +++ b/drivers/base/core.c @@ -9,6 +9,7 @@ */ =20 #include +#include #include #include #include @@ -46,6 +47,65 @@ static bool fw_devlink_drv_reg_done; static bool fw_devlink_best_effort; static struct workqueue_struct *device_link_wq; =20 +enum async_device_shutdown_enabled { + ASYNC_DEV_SHUTDOWN_DISABLED, + ASYNC_DEV_SHUTDOWN_SAFE, + ASYNC_DEV_SHUTDOWN_ENABLED, +}; + +static enum async_device_shutdown_enabled + async_device_shutdown_enabled =3D ASYNC_DEV_SHUTDOWN_SAFE; + +static ssize_t async_device_shutdown_show(struct kobject *kobj, + struct kobj_attribute *attr, char *buf) +{ + const char *output; + + switch (async_device_shutdown_enabled) { + case ASYNC_DEV_SHUTDOWN_DISABLED: + output =3D "off"; + break; + case ASYNC_DEV_SHUTDOWN_SAFE: + output =3D "safe"; + break; + case ASYNC_DEV_SHUTDOWN_ENABLED: + output =3D "on"; + break; + default: + output =3D "unknown"; + } + + return sysfs_emit(buf, "%s\n", output); +} + +static ssize_t async_device_shutdown_store(struct kobject *kobj, + struct kobj_attribute *attr, + const char *buf, size_t count) +{ + if (!capable(CAP_SYS_BOOT)) + return -EPERM; + + if (!strncmp(buf, "off", 3)) + async_device_shutdown_enabled =3D ASYNC_DEV_SHUTDOWN_DISABLED; + else if (!strncmp(buf, "safe", 4)) + async_device_shutdown_enabled =3D ASYNC_DEV_SHUTDOWN_SAFE; + else if (!strncmp(buf, "on", 2)) + async_device_shutdown_enabled =3D ASYNC_DEV_SHUTDOWN_ENABLED; + else + return -EINVAL; + + return count; +} + +static struct kobj_attribute async_device_shutdown_attr =3D __ATTR_RW(asyn= c_device_shutdown); + +static int __init async_shutdown_sysfs_init(void) +{ + return sysfs_create_file(kernel_kobj, &async_device_shutdown_attr.attr); +} + +late_initcall(async_shutdown_sysfs_init); + /** * __fwnode_link_add - Create a link between two fwnode_handles. * @con: Consumer end of the link. @@ -3569,6 +3629,7 @@ static int device_private_init(struct device *dev) klist_init(&dev->p->klist_children, klist_children_get, klist_children_put); INIT_LIST_HEAD(&dev->p->deferred_probe); + dev->p->shutdown_after =3D 0; return 0; } =20 @@ -4819,6 +4880,23 @@ int device_change_owner(struct device *dev, kuid_t k= uid, kgid_t kgid) } EXPORT_SYMBOL_GPL(device_change_owner); =20 +static ASYNC_DOMAIN(sd_domain); + +static bool async_shutdown_allowed(struct device *dev) +{ + if (!dev->driver) + return false; + + switch (async_device_shutdown_enabled) { + case ASYNC_DEV_SHUTDOWN_ENABLED: + return !(dev->driver->shutdown_type =3D=3D SHUTDOWN_FORCE_SYNCHRONOUS); + case ASYNC_DEV_SHUTDOWN_SAFE: + return (dev->driver->shutdown_type =3D=3D SHUTDOWN_PREFER_ASYNCHRONOUS); + default: + return false; + } +} + static void shutdown_one_device(struct device *dev) { /* hold lock to avoid race with probe/release */ @@ -4854,12 +4932,30 @@ static void shutdown_one_device(struct device *dev) put_device(dev->parent); } =20 +/** + * shutdown_one_device_async + * @data: the pointer to the struct device to be shutdown + * @cookie: not used + * + * Shuts down one device, after waiting for dev's shutdown_after to + * complete first. + */ +static void shutdown_one_device_async(void *data, async_cookie_t cookie) +{ + struct device *dev =3D data; + + async_synchronize_cookie_domain(dev->p->shutdown_after + 1, &sd_domain); + + shutdown_one_device(dev); +} + /** * device_shutdown - call ->shutdown() on each device to shutdown. */ void device_shutdown(void) { struct device *dev, *parent; + async_cookie_t cookie =3D 0; =20 wait_for_device_probe(); device_block_probing(); @@ -4890,11 +4986,42 @@ void device_shutdown(void) list_del_init(&dev->kobj.entry); spin_unlock(&devices_kset->list_lock); =20 - shutdown_one_device(dev); + if (async_device_shutdown_enabled) { + struct device_link *link; + int idx; + + /* + * Wait for previous device to shut down if synchronous + */ + if (!async_shutdown_allowed(dev)) + dev->p->shutdown_after =3D cookie; + + get_device(dev); + get_device(parent); + + cookie =3D async_schedule_domain(shutdown_one_device_async, + dev, &sd_domain); + /* + * Ensure parent & suppliers wait for this device to shut down + */ + if (parent) { + parent->p->shutdown_after =3D cookie; + put_device(parent); + } + + idx =3D device_links_read_lock(); + list_for_each_entry_rcu(link, &dev->links.suppliers, c_node, + device_links_read_lock_held()) + link->supplier->p->shutdown_after =3D cookie; + device_links_read_unlock(idx); + put_device(dev); + } else + shutdown_one_device(dev); =20 spin_lock(&devices_kset->list_lock); } spin_unlock(&devices_kset->list_lock); + async_synchronize_full_domain(&sd_domain); } =20 /* diff --git a/include/linux/device/driver.h b/include/linux/device/driver.h index 7738f458995f..f414c8a6f814 100644 --- a/include/linux/device/driver.h +++ b/include/linux/device/driver.h @@ -48,6 +48,12 @@ enum probe_type { PROBE_FORCE_SYNCHRONOUS, }; =20 +enum shutdown_type { + SHUTDOWN_DEFAULT_STRATEGY, + SHUTDOWN_PREFER_ASYNCHRONOUS, + SHUTDOWN_FORCE_SYNCHRONOUS, +}; + /** * struct device_driver - The basic device driver structure * @name: Name of the device driver. @@ -56,6 +62,7 @@ enum probe_type { * @mod_name: Used for built-in modules. * @suppress_bind_attrs: Disables bind/unbind via sysfs. * @probe_type: Type of the probe (synchronous or asynchronous) to use. + * @shutdown_type: Type of the shutdown (synchronous or asynchronous) to u= se. * @of_match_table: The open firmware table. * @acpi_match_table: The ACPI match table. * @probe: Called to query the existence of a specific device, @@ -102,6 +109,7 @@ struct device_driver { =20 bool suppress_bind_attrs; /* disables bind/unbind via sysfs */ enum probe_type probe_type; + enum shutdown_type shutdown_type; =20 const struct of_device_id *of_match_table; const struct acpi_device_id *acpi_match_table; --=20 2.39.3 From nobody Tue Feb 10 23:53:30 2026 Received: from mail-oo1-f50.google.com (mail-oo1-f50.google.com [209.85.161.50]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 25A1014F9E2 for ; Thu, 16 May 2024 15:49:45 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.161.50 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1715874587; cv=none; b=gbaVG0QTycs2CgJyBt906az0qISrx2kXChJGlLZu5Dn9vE6RDNLHEprDxdm7WVSVdexA6KjTwiIgyZOdykZMtfjdMiyyzZqyiBvolh3q2wykU9ymRWN7F456rfsnj68z7SLhKJBrTlmHBfotN3fM06bGsNI02WUGVf7JyJTHlcE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1715874587; c=relaxed/simple; bh=o93R5YZmPt9iBdQ5nObXxgaQGjXN5Y67BVG3ZtpNzQg=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=Ey6m53eJoCLs86bGkX7QnlAHdiMDsKK7N2Lc/GRwn25+3DV1Uru0xvmnD/tD2vJ7UZck7OGWjXqIS3/M5AE328qsd0tOGuvyqZUOmOFjc8SIHwVmWEq0SpavrSMxsxqSU4p+2trw+sjCF7bpye+mc177xlRbOhAKfFgwQx939QA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=N6YTvHwF; arc=none smtp.client-ip=209.85.161.50 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="N6YTvHwF" Received: by mail-oo1-f50.google.com with SMTP id 006d021491bc7-5b2a66dce8fso273130eaf.1 for ; Thu, 16 May 2024 08:49:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1715874585; x=1716479385; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=1el02iUk0200SGhOLa3Aie872dT1lHaSMV4HS5bPfTE=; b=N6YTvHwFYODCBbgk1naSPQ1vrDQAHzN29ZqVDKTD9d5XufW2k+2U3nQFkqQYpGcQcG NFKuOBhpL+sSncr0XOYn3dC6viImxxdU/haxSNv60xM1F37eMOgidjQDUiY9A2h3XXGb 0ZL3rUuuw3nN+pddqLol5CdWpVzbrP0rfH7yE2/bXmI7zTF500pQwvA6MhmpPcBvesYj 1rADV+FDnhdJMJGLaDZBGWEdVTwrnU343L/plBgGtj2aKgqyio6dLmhyOtfxrjADnVba wAglhd2AdNTNgBkgxc0uXFkpNSQVSyTFfsHS11D2OE2QIBM3aced0DNHPFMOpI1UAFJh bycg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1715874585; x=1716479385; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=1el02iUk0200SGhOLa3Aie872dT1lHaSMV4HS5bPfTE=; b=S/CwfwZsWJQoZlJSjuLX5U2Mtxy0DTbJWC0zc11xq84geFDxzcNDo9S91k6zalWkoM 0Vk8V/UAh18Y7cqjZtuMdkVgmYsha9JlzZXK2lSiD6qAVG8ZanQEjc0vJ7xTyOnwxPk9 XeX+fe5BcpBSVL7pfZTmMr/QmA2OGyjolJm9i33E3SEIbpTjpJ2v13Wpq53egxCuppGT tzmCtzso2s7Kx8XVm2urxAyU54IDVUZ+YDZhmjKSwOnyOXowUgKmZd6QPnkNjZw16tN2 nd1ox/VHqA1OITNUiHD7VBsVGAnVSUMZyiF5zJRsnwEwsXGWNnnDkiSqW+RZ4Wr3LmwA 4nxQ== X-Gm-Message-State: AOJu0YyYdTRotZBHyWz1LatqIT8v02YnnE2glLxQkb63dEejQA0Q4wvL CdY4xqKJs/Iu3NvRVkvvBb6p3HR8+MEo1HXBXGHeFuZTH4RGUXN3C8U13qg5 X-Google-Smtp-Source: AGHT+IFWbadnjHt9POkdmHGXGH0+QHLuxmZCux8kZhdMRGGI66+uLqte4Wr/PibsSmfHW7ga2az1Pw== X-Received: by 2002:a05:6870:1715:b0:240:f3ee:6fd8 with SMTP id 586e51a60fabf-24172be01aamr22432820fac.43.1715874584773; Thu, 16 May 2024 08:49:44 -0700 (PDT) Received: from localhost.localdomain ([143.166.81.254]) by smtp.gmail.com with ESMTPSA id 586e51a60fabf-2412a3c82edsm3562563fac.12.2024.05.16.08.49.43 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 16 May 2024 08:49:44 -0700 (PDT) From: Stuart Hayes To: linux-kernel@vger.kernel.org, Greg Kroah-Hartman , "Rafael J . Wysocki" , Tanjore Suresh , Martin Belanger , Oliver O'Halloran , Daniel Wagner , Keith Busch , Lukas Wunner , David Jeffery , Jeremy Allison , Jens Axboe , Christoph Hellwig , Sagi Grimberg , linux-nvme@lists.infradead.org Cc: Stuart Hayes Subject: [PATCH v6 4/4] nvme-pci: Make driver prefer asynchronous shutdown Date: Thu, 16 May 2024 10:49:20 -0500 Message-Id: <20240516154920.221445-5-stuart.w.hayes@gmail.com> X-Mailer: git-send-email 2.39.3 In-Reply-To: <20240516154920.221445-1-stuart.w.hayes@gmail.com> References: <20240516154920.221445-1-stuart.w.hayes@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Set the driver default to prefer asynchronous shutdown. Signed-off-by: Stuart Hayes Signed-off-by: David Jeffery --- drivers/nvme/host/pci.c | 1 + 1 file changed, 1 insertion(+) diff --git a/drivers/nvme/host/pci.c b/drivers/nvme/host/pci.c index 710043086dff..df7bd88f50f2 100644 --- a/drivers/nvme/host/pci.c +++ b/drivers/nvme/host/pci.c @@ -3530,6 +3530,7 @@ static struct pci_driver nvme_driver =3D { .shutdown =3D nvme_shutdown, .driver =3D { .probe_type =3D PROBE_PREFER_ASYNCHRONOUS, + .shutdown_type =3D SHUTDOWN_PREFER_ASYNCHRONOUS, #ifdef CONFIG_PM_SLEEP .pm =3D &nvme_dev_pm_ops, #endif --=20 2.39.3