From nobody Mon Feb 9 03:17:27 2026 Received: from mail-pl1-f194.google.com (mail-pl1-f194.google.com [209.85.214.194]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9FA2A176FA0 for ; Wed, 21 Aug 2024 23:06:09 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.194 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1724281570; cv=none; b=e72Qw0PmSxK/BrR8JYgQVvscfgvWUJ/PeAU0h4uKOgqOa1SupmNBLZp2OnFstWo7d8wM9kfs53PXZ3BkiPY+RhO+pb/gNBFAGg6x5KOsCy8jRGiPrZKTcx18vzRQoPk7+SBrB+ffhD3oXiZe0LMIBI30/WH56Rg10rodZL4n0rg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1724281570; c=relaxed/simple; bh=jzrXL2/Wc/gnh+jcFVlQt7TwdpbKVZ8MiMKPdu9H99U=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=OAH0UEpOCZUwxCLhOybSAIwbBhym8m4qHrljhjfgEV0v7/qCVdvB1t1dSIXAlEbDSgRxTn8lFvQ93civt1YFCHJ2Df5l1gbv+BdaEMbYvBT/q5hEIHiiCtKztqUkHYPndhxcRHfly2Cva1WLrPWWS2KdYDKC7NcqhUhpuF0gBEA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=bQyGdj+E; arc=none smtp.client-ip=209.85.214.194 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="bQyGdj+E" Received: by mail-pl1-f194.google.com with SMTP id d9443c01a7336-2021a99af5eso1824215ad.1 for ; Wed, 21 Aug 2024 16:06:09 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1724281569; x=1724886369; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=vKUMIFDlDxML5PfSkH8a/brAuUCM0XB6TZRLngrEIFo=; b=bQyGdj+EclBpi48379r6UvOmlqQ+UcNLTrvs+R7TFUV+k/ZxuhxcPoFf8CBRk+K8ky thl5NWlol+ZteKEuFY8fxAA2RO+bdbH24bFeHO0o4X436qNbpadhcQjd+eL42LsYdh1J uMTiOg1B2HgnTFHGTP/biwBrWY8f0IlFNRQFNhHe8oMubBkodlEM/rqGxxm5smzU1P+R oOwDuu86gPEWrXX5ObgwWTZOZzlGbNSR6YmOqSo6wQzEaU0sLlRmNcRUE9SCDS3erXf8 bdKj43wyvt8SiYFx0JY08JRqAP/Zbro4yBgNX67x+YsNJ3VDz/84QRJTGUwjajEa/kMv e8vw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1724281569; x=1724886369; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=vKUMIFDlDxML5PfSkH8a/brAuUCM0XB6TZRLngrEIFo=; b=MRfsSKP+HXyWRFzONnAjL1wBIaKbcoCbuJlMulbElfktnlLiLT/Kt9YPmy4AYjP7zh nEonLi3c/W70qLdwVAtWgKXm7ctO0pWzdPGZSIJ/aIk7gABJ8Q6YMNT6JUVpE3VRpQKw kkz2ZCGaF575k45PMuqTAUZyIn0ZPhGyc8mcBomWcdkwT9pDvv4TNu3IcGPP1JlseCYW rakDdtVdB8CXwUloAapoX4btRV8OHmhHqb92jYClUJAqqBtT2+4UygnryswU3mcyvO70 8lhKM04rgSQqwEYiqP+3U5amXb0la0ClsPpPTb2kQbCEBsHG9r2de3ezYGxLP7t0dEqS yPTw== X-Gm-Message-State: AOJu0YyO6xCMVQVuo3YBmfsx8A08CNAujdbt5g9fmXLlj12A3W1lmwk5 cUe7rd1KKNspyzl8dFlsYL/TO0sfBVZnFbyNxZuSi0+qDF148Nnv X-Google-Smtp-Source: AGHT+IEIneXRyLEnHTQM+Mgu7X5rFgB0b/SNpcbcwYF5eqGbonCqHaHsn/QxfJsqm3Si1Ya41ZmcTg== X-Received: by 2002:a17:902:cecd:b0:202:2d:c87 with SMTP id d9443c01a7336-2038820fb1dmr1438735ad.12.1724281568671; Wed, 21 Aug 2024 16:06:08 -0700 (PDT) Received: from localhost (192.243.127.228.16clouds.com. [192.243.127.228]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-20385566509sm1182065ad.36.2024.08.21.16.06.08 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 21 Aug 2024 16:06:08 -0700 (PDT) From: Woody Zhang To: Thomas Gleixner , Andrew Morton Cc: linux-kernel@vger.kernel.org, Woody Zhang Subject: [PATCH 1/5] bit_spinlock: add irq variant for bit spinlock API Date: Thu, 22 Aug 2024 07:05:35 +0800 Message-ID: <20240821230539.168107-2-woodyzhang666@gmail.com> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20240821230539.168107-1-woodyzhang666@gmail.com> References: <20240821230539.168107-1-woodyzhang666@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" This variant makes bit spinlock easy to be used to protect data that can be accessed from irq context. Signed-off-by: Woody Zhang --- include/linux/bit_spinlock.h | 37 ++++++++++++++++++++++++++++++++++++ 1 file changed, 37 insertions(+) diff --git a/include/linux/bit_spinlock.h b/include/linux/bit_spinlock.h index bbc4730a6505..5a176e574a8f 100644 --- a/include/linux/bit_spinlock.h +++ b/include/linux/bit_spinlock.h @@ -6,6 +6,9 @@ #include #include #include +#include +#include +#include =20 /* * bit-based spin_lock() @@ -97,5 +100,39 @@ static inline int bit_spin_is_locked(int bitnum, unsign= ed long *addr) #endif } =20 +#define bit_spin_lock_irqsave(bitnum, addr, flags) \ +do { \ + typecheck(int, bitnum); \ + typecheck(unsigned long *, addr); \ + typecheck(unsigned long, flags); \ + local_irq_save(flags); \ + bit_spin_lock(bitnum, addr); \ +} while (0) + + +#define bit_spin_trylock_irqsave(bitnum, addr, flags) \ +({ \ + typecheck(int, bitnum); \ + typecheck(unsigned long *, addr); \ + typecheck(unsigned long, flags); \ + local_irq_save(flags); \ + bit_spin_trylock(bitnum, addr) ? \ + 1 : ({ local_irq_restore(flags); 0; }); \ +}) + +static inline void bit_spin_unlock_irqrestore(int bitnum, + unsigned long *addr, unsigned long flags) +{ + bit_spin_unlock(bitnum, addr); + local_irq_restore(flags); +} + +static inline void __bit_spin_unlock_irqrestore(int bitnum, + unsigned long *addr, unsigned long flags) +{ + __bit_spin_unlock(bitnum, addr); + local_irq_restore(flags); +} + #endif /* __LINUX_BIT_SPINLOCK_H */ =20 --=20 2.45.2 From nobody Mon Feb 9 03:17:27 2026 Received: from mail-pf1-f193.google.com (mail-pf1-f193.google.com [209.85.210.193]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 4241817A924 for ; Wed, 21 Aug 2024 23:06:19 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.193 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1724281580; cv=none; b=HOThro8hyiEseQupwz+RP6qjzdEml/ZmgVO2B00BY8xzWhXn7S2e6Y2kAw87C+cQY6UjU7otp0IxMRFMVV3gj3HvJ2mVaDEM3fbG1LVH0NQFdvs4XJYSXnSn+9lCVhrrTKV9ePhxIy6pQve+HYfIuP9iEU2sawyxJUNT5uCvv9A= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1724281580; c=relaxed/simple; bh=6hNk9e2UJVZwlstXMPYh1DVJSpZvzE7EkgsrAtnLwXM=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=qxdPF3N6WIHP4AArY2dYS3lgf6jYyGJwm2FJGFAsyVSsCXS2nedDq41j+zKRPjvutBhro9fWqSCgaf0SOmuq4JupTSfh16x+7S3wUHRw3XDfySsDiTlM9WnLc+ZivlatGkkJJ/ZkJwSH0FKrIfIpB5lccSygh0MOGkCQ9A4WSAA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=FtEeogBO; arc=none smtp.client-ip=209.85.210.193 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="FtEeogBO" Received: by mail-pf1-f193.google.com with SMTP id d2e1a72fcca58-7106cf5771bso193711b3a.2 for ; Wed, 21 Aug 2024 16:06:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1724281578; x=1724886378; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=6JYgy2mO7B9WkjZc16hmBlriFY5uzQgt+dUrYiCZERg=; b=FtEeogBO0kI+ExZVGiCB0j85lGJzE4jYlsrXZWSN4oyMpL38VQbZZhxMZXvtdikz/Z CgUfSMPs4jkfGFmXq2pKGGTn8vZvt+AD15JfLNcpexKcA/HpB5qtxfWF+1bRPA/3QkIt iodwA5TiIZCH9R5PA61ylIQHKh2j2xl7nvR4+TUebLXtO2tEUOMFCT/sA/7rm3cUrXeQ WhrChM5eis7Sycjh6c1+8gDgxJ4qmDuOP6Al7+3y4AUnBfYRSYq8ax03cQuoF7WzJRFh rkemHZRzCd1C+sq1yVCcpUgaddrPmjg9bapqA/CCsHqW3dLhK/WipbWjS/QoGizDE323 9I6g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1724281578; x=1724886378; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=6JYgy2mO7B9WkjZc16hmBlriFY5uzQgt+dUrYiCZERg=; b=EW2reZlt1eN3PJ3yOYI8+IftDS9DEbsSpqAuTD36yv13HwLuYmz5pqVGO1ebQ+DV5h rplRWr6DLCB8URvVg8cNmPBleSlJbMFN185EYMWbaHSPsaGjpWwjHKuRHeT3Hhr/tYPg Ij00b0fRz7ALjMmnsPaSf9pjsU5LumMTagkQBFRvABBFL+LnmqMLGp/7O+b8hHyLIaO+ uvrgyTJj5HKN8OACeINRXQQTnZCrSIqNZqVwV1NI/QckxlCMubLb6p92/n8XT2EBibPZ Bm7HG9+mapKbfTlGf3dcgO4IxvKLSADfKOUyA+ZRni3OwmlBAU7QXyF98k7RkXTFH4+I umAg== X-Gm-Message-State: AOJu0YxXy4KFqqaGDN85cXPI0PUXKCLbOrKMAildr3hodMHyTJprvfYw bpzuVLiqfluO3RUKwKggFkat96tuCMuso4J5iU02FCAirv7/7HZX X-Google-Smtp-Source: AGHT+IHWeoiz+aECf92Ri7281VwEPf/0I+K7OqHjM4FcrWKptmVsHw/fALssf8mlgiaZBXtoSoStsg== X-Received: by 2002:a05:6a00:1898:b0:714:2cea:1473 with SMTP id d2e1a72fcca58-7142cea15bfmr2722751b3a.23.1724281578259; Wed, 21 Aug 2024 16:06:18 -0700 (PDT) Received: from localhost (192.243.127.228.16clouds.com. [192.243.127.228]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-714343972e3sm170889b3a.139.2024.08.21.16.06.17 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 21 Aug 2024 16:06:18 -0700 (PDT) From: Woody Zhang To: Thomas Gleixner , Andrew Morton Cc: linux-kernel@vger.kernel.org, Woody Zhang Subject: [PATCH 2/5] list_bl: add irq variant for hlist_bl lock API Date: Thu, 22 Aug 2024 07:05:36 +0800 Message-ID: <20240821230539.168107-3-woodyzhang666@gmail.com> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20240821230539.168107-1-woodyzhang666@gmail.com> References: <20240821230539.168107-1-woodyzhang666@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" This variant makes it easy to be used to protect hash list that can be accessed from irq context. Signed-off-by: Woody Zhang --- include/linux/list_bl.h | 21 +++++++++++++++++++++ 1 file changed, 21 insertions(+) diff --git a/include/linux/list_bl.h b/include/linux/list_bl.h index ae1b541446c9..7ce411567fe5 100644 --- a/include/linux/list_bl.h +++ b/include/linux/list_bl.h @@ -153,6 +153,27 @@ static inline void hlist_bl_unlock(struct hlist_bl_hea= d *b) __bit_spin_unlock(0, (unsigned long *)b); } =20 +#define hlist_bl_lock_irqsave(b, flags) \ +do { \ + typecheck(struct hlist_bl_head*, b); \ + typecheck(unsigned long, flags); \ + bit_spin_lock_irqsave(0, (unsigned long *)b, flags); \ +} while (0) + +#define hlist_bl_trylock_irqsave(b, flags) \ +({ \ + typecheck(struct hlist_bl_head*, b); \ + typecheck(unsigned long, flags); \ + bit_spin_trylock_irqsave(0, (unsigned long *)b, flags); \ +}) + +static inline void hlist_bl_unlock_irqrestore(struct hlist_bl_head *b, + unsigned long flags) +{ + __bit_spin_unlock_irqrestore(0, (unsigned long *)b, flags); +} + + static inline bool hlist_bl_is_locked(struct hlist_bl_head *b) { return bit_spin_is_locked(0, (unsigned long *)b); --=20 2.45.2 From nobody Mon Feb 9 03:17:27 2026 Received: from mail-pf1-f196.google.com (mail-pf1-f196.google.com [209.85.210.196]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 0DD4317C7C7 for ; Wed, 21 Aug 2024 23:06:25 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.196 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1724281587; cv=none; b=Ek99iXQPvtxxBGCRSCGA+7OPFHBkzkFYNERBj1YPF7eIXKNjkcqO5V14XgOWBScRcSmchFvHAALhAG4u8YSNoPER17/900MB7nWwUBrrXOWtGiskbn5wH5m6cYcPNoT1h6ciTpFujdiI1x2fLkY8s4rGOFMyI0gIJGiMwlO8Mz8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1724281587; c=relaxed/simple; bh=+2a3e6fzU6g1LxfJMSDIChbvuKERgiPIByYdrnn6FHw=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=UezEZzrxDskic0Ln9AuMPBickYKg371jQwub1UDJ34RbP2J1wy62sutAn/9yTWLoE2HGDDkA2bmjk4CGXAXJYiymN5eiygse7FWDOzBQTE9Se5FCZu5MSBvR9s3nuvfDPGBwx3yVBOzW+HbZSjPHPW0R8gJL9aHrLa/StZc6jck= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=CMNAq1Kz; arc=none smtp.client-ip=209.85.210.196 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="CMNAq1Kz" Received: by mail-pf1-f196.google.com with SMTP id d2e1a72fcca58-713eeb4e4a9so176443b3a.0 for ; Wed, 21 Aug 2024 16:06:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1724281585; x=1724886385; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=+j8AVcy3nA+u6dMBdfV0u6O2tPyGN+umcRjrnsJwbL8=; b=CMNAq1Kza+otw96LXm97pwhLUG5GWtBfuIGqlDAdIgRBy5OcTOHzeDAFQ1+9Krgt5n o31c7ZiPqIoFAWcn0xHK9pqczHDIMLsIx9w+1aHpA+WSQiTnf9QvUvC1ECmA5/1jQP1T JhtDiNbtaUTUGd6tF69tkfWvmgXh4sU04yB0cCV6suGMok2ySMlDucsrkoHsEcnMUIkr 5XcjC46cLKvvjpLxGPkK+VyDNNWEkRMVDpcuTwefUP99ek9cSg4CzV/hapmykrRmEt2O hhb96jeWikNBr1sdcQQI1p3gUGaOKeccyfjYBQCWGaum/LU1apLn4VT7Bv39vsdfQigJ Wq+g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1724281585; x=1724886385; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=+j8AVcy3nA+u6dMBdfV0u6O2tPyGN+umcRjrnsJwbL8=; b=SUQ5eyNvu7mPjdyROFCwzoNTzpQ3Jhjd5alkg8H1bQTMEJP4dc9xx1zteAzFeeWG1h yurTRLp8ra0uCbNt8b55pK/CPUJMn2unu6UtPic2EFRIX70rheUa7RYXOTgm3bC4a0XS SgGoRIGQIVzoK6Xp+MNb7GX15OdcpspF7IonBkj4rqdMple2GxWpGBk6K4pQ8qluoFm9 IKm3TbEO7MpqfQ8feCc40RjU9LyGBBNsKLWLFV0bgcYKUoNIiqCCeWLsLfDVx4NwL3o8 6tJXIpSeGcxEtbbOkBRFzz+OMJBZdSmZ+kKCuj9ENIpMo3GX4gpHYVTb6cAa+S9BSNNM FOdA== X-Gm-Message-State: AOJu0YxmG+PwHoSX6x6luVgKu3gbMMAgSJ2BSl0PnBSeyfV/LCzNTk9n BJiVS5hxQPlc018CWSvEanJj7NsRGYuhe1286AaYYxBK/Y4/GUsG X-Google-Smtp-Source: AGHT+IFfBczXZo+UWU5F8+qqVF5zKafTj4LZ8XIO00fuY/ucWwm22VoBvTClV6atKCO0mLVyIR6r/Q== X-Received: by 2002:a05:6a00:10c4:b0:70d:1b17:3c5e with SMTP id d2e1a72fcca58-71423448b4dmr4619860b3a.6.1724281585084; Wed, 21 Aug 2024 16:06:25 -0700 (PDT) Received: from localhost (192.243.127.228.16clouds.com. [192.243.127.228]) by smtp.gmail.com with ESMTPSA id 41be03b00d2f7-7cd9ad7db5fsm111843a12.92.2024.08.21.16.06.24 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 21 Aug 2024 16:06:24 -0700 (PDT) From: Woody Zhang To: Thomas Gleixner , Andrew Morton Cc: linux-kernel@vger.kernel.org, Woody Zhang Subject: [PATCH 3/5] list_bl: remove lock check in hlist_bl_set_first Date: Thu, 22 Aug 2024 07:05:37 +0800 Message-ID: <20240821230539.168107-4-woodyzhang666@gmail.com> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20240821230539.168107-1-woodyzhang666@gmail.com> References: <20240821230539.168107-1-woodyzhang666@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" This allows it to be used in situations when locking is not required, e.g. in early initialization phase. Signed-off-by: Woody Zhang --- include/linux/list_bl.h | 7 +++---- 1 file changed, 3 insertions(+), 4 deletions(-) diff --git a/include/linux/list_bl.h b/include/linux/list_bl.h index 7ce411567fe5..d25c659f6635 100644 --- a/include/linux/list_bl.h +++ b/include/linux/list_bl.h @@ -63,10 +63,9 @@ static inline struct hlist_bl_node *hlist_bl_first(struc= t hlist_bl_head *h) static inline void hlist_bl_set_first(struct hlist_bl_head *h, struct hlist_bl_node *n) { - LIST_BL_BUG_ON((unsigned long)n & LIST_BL_LOCKMASK); - LIST_BL_BUG_ON(((unsigned long)h->first & LIST_BL_LOCKMASK) !=3D - LIST_BL_LOCKMASK); - h->first =3D (struct hlist_bl_node *)((unsigned long)n | LIST_BL_LOCKMASK= ); + LIST_BL_BUG_ON((uintptr_t)n & LIST_BL_LOCKMASK); + h->first =3D (struct hlist_bl_node *) + ((uintptr_t)n | ((uintptr_t)h->first & LIST_BL_LOCKMASK)); } =20 static inline bool hlist_bl_empty(const struct hlist_bl_head *h) --=20 2.45.2 From nobody Mon Feb 9 03:17:27 2026 Received: from mail-oa1-f65.google.com (mail-oa1-f65.google.com [209.85.160.65]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9B2AE17C9EA for ; Wed, 21 Aug 2024 23:06:31 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.160.65 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1724281593; cv=none; b=k55iI5eeleXrw0Qd2NLI9Vmwu6BLX8Td9CtUJnfTB2qAWMSoaGdOerYPXPkSCvdL9kIxbQmLAbiFQ2FRd7mX5g969DDZ9mKbENlDkA9s9HgH/0H3CVwx/3Lvi9mjVdTrS6ilgsgW2Gq2PEytgOiBi76j0YxJa4FiG110Dx7mad8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1724281593; c=relaxed/simple; bh=Vkd90pptKLQx4ViwZfakrmJMkdGWpS2Kbx0GQlsRk7w=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=WUPmUDD5BgGk5UAOr1mn7bV75kMOQaEIH+k9HNkEMg4F8Rg+oWF/zx6/6PxfqNjCdmrs3CBo28RBi8HYqxFEZH2DrtkWi40i52OrDLpCUMx3Bqzl1FHnPGKU1n5bLPmJQq9KfFozkISDPK2uwZwjImPOg35T45lZ5slmJedsTKs= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=CFOuBSPL; arc=none smtp.client-ip=209.85.160.65 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="CFOuBSPL" Received: by mail-oa1-f65.google.com with SMTP id 586e51a60fabf-27018df4ff3so135706fac.1 for ; Wed, 21 Aug 2024 16:06:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1724281590; x=1724886390; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=bMobP6ENMIF18aN0lwW15eW325zdEEMUYT1+nFph7nc=; b=CFOuBSPLVuFbY1rxE6KvSdBa5RM/S3949uwFYUFMBJ2WlBgYToTv29NhGmGZzQ0EBM KUyWd7ZhkSxkOwWvvOG+MDSsXjFxZgCUMY7KoAwkwcZsChTh/f4+gGuIjRJOUMwqG0Tm 9MjG+uq+5Gx3nHX/0zISlTa033+8w/L/dW9XIByIPKXsJFOkimWfW8OqS+vx5WN0oglC 308PKOwbvsvza5jv4uzl/AHqaqpUDBU/awxFPHcvR4N4zVDSV6vNEyHJggCySZOzG63d VOG+HFyRJ19aVCzeQiQX8Z9IOzmxek9OGzjSxoPktsA8KniKwqTytYCk1BR2dkHIMcj3 WFdw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1724281590; x=1724886390; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=bMobP6ENMIF18aN0lwW15eW325zdEEMUYT1+nFph7nc=; b=tfcSPoXsY9m8HKXkg7NZ/6kjHKytmYDvJghIAhDo5nXEK+CAs3tmy3I+UXcLWQpG0Z XMQjguh/y9o/dHXC7hzDyVNSCltkUiUygwbTUzR/mviZaSrpayrQzbOP8hJYqdg1K8Pm wK1Vma5aJUVKjPOfcbHjVItXAwdThi4qMM5Iqmr+Ad56ES4iiHpbxXy+Kg10UqsA+GSt SvRjoILcw27aAHlrDT0VzTnJlJDVRGhHwu3CEAmL4ckiBwWpCKU6HqEWFfcbrfduWo3f 2ojWz/jSEWba/dAwlKlzOb/oR+Ok4eqaRU9bJZn3x3UEh8Aue0BBmAWQRPBFffEjS3Gy nJ5g== X-Gm-Message-State: AOJu0YwdU57LO1fMIZk8Q9x315hQo+WtypQOjAnWwRxMPvurkRbHwDVT I0BUDF3QzIu6mPlqyenBAXUHx330hqzLgx2nOzkFHEEK2BoGVilC X-Google-Smtp-Source: AGHT+IFAs+iNBIXDAB45Avh+la8ZO7s3H2bqsrk2GdeLbH9coOTybgO/Jil0qiCzBKLvps72K2SBNg== X-Received: by 2002:a05:6870:9127:b0:25e:d62:f297 with SMTP id 586e51a60fabf-273cff95d3amr154177fac.45.1724281590546; Wed, 21 Aug 2024 16:06:30 -0700 (PDT) Received: from localhost (192.243.127.228.16clouds.com. [192.243.127.228]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-71434235f77sm184096b3a.13.2024.08.21.16.06.30 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 21 Aug 2024 16:06:30 -0700 (PDT) From: Woody Zhang To: Thomas Gleixner , Andrew Morton Cc: linux-kernel@vger.kernel.org, Woody Zhang Subject: [PATCH 4/5] list_bl: add hlist_bl_move_list and two macros Date: Thu, 22 Aug 2024 07:05:38 +0800 Message-ID: <20240821230539.168107-5-woodyzhang666@gmail.com> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20240821230539.168107-1-woodyzhang666@gmail.com> References: <20240821230539.168107-1-woodyzhang666@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" hlist_bl_move_list is similar to hlist_move_list. Caller should decide whether bit locking is required for old and new hash list. Signed-off-by: Woody Zhang --- include/linux/list_bl.h | 13 +++++++++++++ 1 file changed, 13 insertions(+) diff --git a/include/linux/list_bl.h b/include/linux/list_bl.h index d25c659f6635..928048354e04 100644 --- a/include/linux/list_bl.h +++ b/include/linux/list_bl.h @@ -38,6 +38,8 @@ struct hlist_bl_head { struct hlist_bl_node { struct hlist_bl_node *next, **pprev; }; +#define HLIST_BL_HEAD_INIT { .first =3D NULL } +#define HLIST_BL_HEAD(name) struct hlist_bl_head name =3D { .first =3D NU= LL } #define INIT_HLIST_BL_HEAD(ptr) \ ((ptr)->first =3D NULL) =20 @@ -142,6 +144,17 @@ static inline void hlist_bl_del_init(struct hlist_bl_n= ode *n) } } =20 +static inline void hlist_bl_move_list(struct hlist_bl_head *old, + struct hlist_bl_head *new) +{ + struct hlist_bl_node *n =3D hlist_bl_first(old); + + hlist_bl_set_first(new, n); + if (n) + n->pprev =3D &new->first; + hlist_bl_set_first(old, NULL); +} + static inline void hlist_bl_lock(struct hlist_bl_head *b) { bit_spin_lock(0, (unsigned long *)b); --=20 2.45.2 From nobody Mon Feb 9 03:17:27 2026 Received: from mail-pf1-f196.google.com (mail-pf1-f196.google.com [209.85.210.196]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D915817CA14 for ; Wed, 21 Aug 2024 23:06:39 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.196 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1724281602; cv=none; b=dCwmDUrQdlnRktHDPl21Ii2ftpza2Ik8duCgyL7QSbJzxaOHJUcNea9RuHwkfGFB6k6h/JqnRi5p62ROehxzcCKwpr31XXr520UWe2iPYRHoyvSXYAd5UQN8wKD2dtSqPSJQVfRwl71yYpbSShrpU6aHeX9VPjygibV6NQaS8K8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1724281602; c=relaxed/simple; bh=XN+rdEQOGamw9jTCOYuF+Hib81g2JNJcluTrSrE7Agw=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=BbpWwoGTSTxyTAcjIO8qgv3p+m90m2U2lmgzO6uFPlkO3Nw0UzhmlNvB1+MZ22JHDuUFqqCg5nPim0NasohmHk0d6flazYoHaFhT0tOzObCJ+UD4tbmaAapkBECCiaoLSRr+jWVpRU8a5JpGjhX/CllIK9zQWsQIYRw2hztOeBI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=BfzzTyqD; arc=none smtp.client-ip=209.85.210.196 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="BfzzTyqD" Received: by mail-pf1-f196.google.com with SMTP id d2e1a72fcca58-714263cb074so206572b3a.1 for ; Wed, 21 Aug 2024 16:06:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1724281599; x=1724886399; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=DNFMTHpoapmJ/WJI+sAnXGyaXgVzeQputyNi83Y8ox0=; b=BfzzTyqDEaf4spr1gzmVvuxDL2lIYvBFojGkaAs7vWcMq2aIYoc8/hwFl9gTHMRi2+ HDkRyU0k/o/3yzV1OW14CxFUk5L5UBtmNLj2N5CUomLYJsQUJFokjEQjMJNhCXbbUuVm yuRTq5BGEvFOpKdyzdlFdSZaErRQJjjNYVUF1Vb9c+LaisoHA6ReyJv3xxZvNHD6sGTr 17Obs4h2sTZ6XKefTUKNanLRjRNGg/6t859hwbmroD7EDObwdRQ6H/EFf+OVWWTkNTPO 1uSIYsmWudLUsz5UjBqZqRqYcOl/n33SXMUtXABy44pFXprOkCtKEk3u9G6am2GdeXLM YskA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1724281599; x=1724886399; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=DNFMTHpoapmJ/WJI+sAnXGyaXgVzeQputyNi83Y8ox0=; b=d3xw2xdhnyPN2m+VrdScPvlqMsdlCefMZdRCxFmAFCmaEykphADvhLTpcmZVy8UTpQ 0/zic+ihkdWYRgnL+Fg2yVztm6Uov6/GUmvNs78QnmYHDG6x1eM2xoTx5fEBbQRWvTsq RBmcyn73auzjQ06PZ+qFfzfMRBZYKEoE3PlDkxUTrolSSKyz+cDwtM6Z30OypjdRDqGG o12nHSx/enUT7XHTzJPwNH+UpRGJll6+E9JyX9ENRJNhmyKLBpMNl13UDplFZELjrEIh aUnWPIcbX7t1q3jSVqpR6baba7HmfYXZTsJyJZnR9GL83X+xp8v6DhcNePqMvungGSvU YwIQ== X-Gm-Message-State: AOJu0YyVGIz/sa/ft5A0CEx1LW87cvv22Holf4NdKm+vmBCt2fRQCKB7 d3DLx70g9lwOSaZPI9oWyrPgs4v59XXC47TNuqhDDCSdkGeetGwh X-Google-Smtp-Source: AGHT+IGoFV1n5PwRxmjQ/GKCkPMrEPUz6mW5eQeI9Nuc0rh3fPHSYKsoRSkr3q2mXTTNvKllP5BzNg== X-Received: by 2002:a05:6a21:3a45:b0:1c4:9e5f:c645 with SMTP id adf61e73a8af0-1caeb30790dmr89701637.40.1724281598798; Wed, 21 Aug 2024 16:06:38 -0700 (PDT) Received: from localhost (192.243.127.228.16clouds.com. [192.243.127.228]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-2d5ebbef1edsm2493325a91.47.2024.08.21.16.06.37 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 21 Aug 2024 16:06:38 -0700 (PDT) From: Woody Zhang To: Thomas Gleixner , Andrew Morton Cc: linux-kernel@vger.kernel.org, Woody Zhang Subject: [PATCH 5/5] debugobjects: use list_bl to save memory for hash slot spinlocks Date: Thu, 22 Aug 2024 07:05:39 +0800 Message-ID: <20240821230539.168107-6-woodyzhang666@gmail.com> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20240821230539.168107-1-woodyzhang666@gmail.com> References: <20240821230539.168107-1-woodyzhang666@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Replace hlist_head + spinlock with hlist_bl_head to save memory for spinlocks. Signed-off-by: Woody Zhang --- include/linux/debugobjects.h | 4 +- lib/debugobjects.c | 288 +++++++++++++++++------------------ 2 files changed, 144 insertions(+), 148 deletions(-) diff --git a/include/linux/debugobjects.h b/include/linux/debugobjects.h index 32444686b6ff..e5b27295d50f 100644 --- a/include/linux/debugobjects.h +++ b/include/linux/debugobjects.h @@ -2,7 +2,7 @@ #ifndef _LINUX_DEBUGOBJECTS_H #define _LINUX_DEBUGOBJECTS_H =20 -#include +#include #include =20 enum debug_obj_state { @@ -26,7 +26,7 @@ struct debug_obj_descr; * @descr: pointer to an object type specific debug description structure */ struct debug_obj { - struct hlist_node node; + struct hlist_bl_node node; enum debug_obj_state state; unsigned int astate; void *object; diff --git a/lib/debugobjects.c b/lib/debugobjects.c index 7cea91e193a8..d68a96e351aa 100644 --- a/lib/debugobjects.c +++ b/lib/debugobjects.c @@ -38,30 +38,23 @@ #define ODEBUG_FREE_WORK_MAX 1024 #define ODEBUG_FREE_WORK_DELAY DIV_ROUND_UP(HZ, 10) =20 -struct debug_bucket { - struct hlist_head list; - raw_spinlock_t lock; -}; - /* * Debug object percpu free list * Access is protected by disabling irq */ struct debug_percpu_free { - struct hlist_head free_objs; + struct hlist_bl_head free_objs; int obj_free; }; =20 static DEFINE_PER_CPU(struct debug_percpu_free, percpu_obj_pool); =20 -static struct debug_bucket obj_hash[ODEBUG_HASH_SIZE]; +static struct hlist_bl_head obj_hash[ODEBUG_HASH_SIZE]; =20 static struct debug_obj obj_static_pool[ODEBUG_POOL_SIZE] __initdata; =20 -static DEFINE_RAW_SPINLOCK(pool_lock); - -static HLIST_HEAD(obj_pool); -static HLIST_HEAD(obj_to_free); +static HLIST_BL_HEAD(obj_pool); +static HLIST_BL_HEAD(obj_to_free); =20 /* * Because of the presence of percpu free pools, obj_pool_free will @@ -139,23 +132,23 @@ static void fill_pool(void) * when allocating. * * Both obj_nr_tofree and obj_pool_free are checked locklessly; the - * READ_ONCE()s pair with the WRITE_ONCE()s in pool_lock critical - * sections. + * READ_ONCE()s pair with the WRITE_ONCE()s in obj_pool bit lock + * critical sections. */ while (READ_ONCE(obj_nr_tofree) && (READ_ONCE(obj_pool_free) < obj_pool_m= in_free)) { - raw_spin_lock_irqsave(&pool_lock, flags); + hlist_bl_lock_irqsave(&obj_pool, flags); /* * Recheck with the lock held as the worker thread might have * won the race and freed the global free list already. */ while (obj_nr_tofree && (obj_pool_free < obj_pool_min_free)) { - obj =3D hlist_entry(obj_to_free.first, typeof(*obj), node); - hlist_del(&obj->node); + obj =3D hlist_bl_entry(obj_to_free.first, typeof(*obj), node); + hlist_bl_del(&obj->node); WRITE_ONCE(obj_nr_tofree, obj_nr_tofree - 1); - hlist_add_head(&obj->node, &obj_pool); + hlist_bl_add_head(&obj->node, &obj_pool); WRITE_ONCE(obj_pool_free, obj_pool_free + 1); } - raw_spin_unlock_irqrestore(&pool_lock, flags); + hlist_bl_unlock_irqrestore(&obj_pool, flags); } =20 if (unlikely(!obj_cache)) @@ -173,25 +166,26 @@ static void fill_pool(void) if (!cnt) return; =20 - raw_spin_lock_irqsave(&pool_lock, flags); + hlist_bl_lock_irqsave(&obj_pool, flags); while (cnt) { - hlist_add_head(&new[--cnt]->node, &obj_pool); + hlist_bl_add_head(&new[--cnt]->node, &obj_pool); debug_objects_allocated++; WRITE_ONCE(obj_pool_free, obj_pool_free + 1); } - raw_spin_unlock_irqrestore(&pool_lock, flags); + hlist_bl_unlock_irqrestore(&obj_pool, flags); } } =20 /* * Lookup an object in the hash bucket. */ -static struct debug_obj *lookup_object(void *addr, struct debug_bucket *b) +static struct debug_obj *lookup_object(void *addr, struct hlist_bl_head *l= ist) { struct debug_obj *obj; + struct hlist_bl_node *tmp; int cnt =3D 0; =20 - hlist_for_each_entry(obj, &b->list, node) { + hlist_bl_for_each_entry(obj, tmp, list, node) { cnt++; if (obj->object =3D=3D addr) return obj; @@ -205,20 +199,22 @@ static struct debug_obj *lookup_object(void *addr, st= ruct debug_bucket *b) /* * Allocate a new object from the hlist */ -static struct debug_obj *__alloc_object(struct hlist_head *list) +static struct debug_obj *__alloc_object(struct hlist_bl_head *list) { struct debug_obj *obj =3D NULL; + struct hlist_bl_node *first =3D hlist_bl_first(list); =20 - if (list->first) { - obj =3D hlist_entry(list->first, typeof(*obj), node); - hlist_del(&obj->node); + if (first) { + obj =3D hlist_bl_entry(first, typeof(*obj), node); + hlist_bl_del(&obj->node); } =20 return obj; } =20 -static struct debug_obj * -alloc_object(void *addr, struct debug_bucket *b, const struct debug_obj_de= scr *descr) +static struct debug_obj *alloc_object(void *addr, + struct hlist_bl_head *list, + const struct debug_obj_descr *descr) { struct debug_percpu_free *percpu_pool =3D this_cpu_ptr(&percpu_obj_pool); struct debug_obj *obj; @@ -231,7 +227,7 @@ alloc_object(void *addr, struct debug_bucket *b, const = struct debug_obj_descr *d } } =20 - raw_spin_lock(&pool_lock); + hlist_bl_lock(&obj_pool); obj =3D __alloc_object(&obj_pool); if (obj) { obj_pool_used++; @@ -250,7 +246,7 @@ alloc_object(void *addr, struct debug_bucket *b, const = struct debug_obj_descr *d obj2 =3D __alloc_object(&obj_pool); if (!obj2) break; - hlist_add_head(&obj2->node, + hlist_bl_add_head(&obj2->node, &percpu_pool->free_objs); percpu_pool->obj_free++; obj_pool_used++; @@ -264,7 +260,7 @@ alloc_object(void *addr, struct debug_bucket *b, const = struct debug_obj_descr *d if (obj_pool_free < obj_pool_min_free) obj_pool_min_free =3D obj_pool_free; } - raw_spin_unlock(&pool_lock); + hlist_bl_unlock(&obj_pool); =20 init_obj: if (obj) { @@ -272,7 +268,7 @@ alloc_object(void *addr, struct debug_bucket *b, const = struct debug_obj_descr *d obj->descr =3D descr; obj->state =3D ODEBUG_STATE_NONE; obj->astate =3D 0; - hlist_add_head(&obj->node, &b->list); + hlist_bl_add_head(&obj->node, list); } return obj; } @@ -285,13 +281,13 @@ alloc_object(void *addr, struct debug_bucket *b, cons= t struct debug_obj_descr *d */ static void free_obj_work(struct work_struct *work) { - struct hlist_node *tmp; + struct hlist_bl_node *tmp, *n; struct debug_obj *obj; unsigned long flags; - HLIST_HEAD(tofree); + HLIST_BL_HEAD(tofree); =20 WRITE_ONCE(obj_freeing, false); - if (!raw_spin_trylock_irqsave(&pool_lock, flags)) + if (!hlist_bl_trylock_irqsave(&obj_pool, flags)) return; =20 if (obj_pool_free >=3D debug_objects_pool_size) @@ -305,13 +301,13 @@ static void free_obj_work(struct work_struct *work) * of them until the next round. */ while (obj_nr_tofree && obj_pool_free < debug_objects_pool_size) { - obj =3D hlist_entry(obj_to_free.first, typeof(*obj), node); - hlist_del(&obj->node); - hlist_add_head(&obj->node, &obj_pool); + obj =3D hlist_bl_entry(hlist_bl_first(&obj_to_free), typeof(*obj), node); + hlist_bl_del(&obj->node); + hlist_bl_add_head(&obj->node, &obj_pool); WRITE_ONCE(obj_pool_free, obj_pool_free + 1); WRITE_ONCE(obj_nr_tofree, obj_nr_tofree - 1); } - raw_spin_unlock_irqrestore(&pool_lock, flags); + hlist_bl_unlock_irqrestore(&obj_pool, flags); return; =20 free_objs: @@ -321,14 +317,14 @@ static void free_obj_work(struct work_struct *work) * memory outside the pool_lock held region. */ if (obj_nr_tofree) { - hlist_move_list(&obj_to_free, &tofree); + hlist_bl_move_list(&obj_to_free, &tofree); debug_objects_freed +=3D obj_nr_tofree; WRITE_ONCE(obj_nr_tofree, 0); } - raw_spin_unlock_irqrestore(&pool_lock, flags); + hlist_bl_unlock_irqrestore(&obj_pool, flags); =20 - hlist_for_each_entry_safe(obj, tmp, &tofree, node) { - hlist_del(&obj->node); + hlist_bl_for_each_entry_safe(obj, tmp, n, &tofree, node) { + hlist_bl_del(&obj->node); kmem_cache_free(obj_cache, obj); } } @@ -350,7 +346,7 @@ static void __free_object(struct debug_obj *obj) */ percpu_pool =3D this_cpu_ptr(&percpu_obj_pool); if (percpu_pool->obj_free < ODEBUG_POOL_PERCPU_SIZE) { - hlist_add_head(&obj->node, &percpu_pool->free_objs); + hlist_bl_add_head(&obj->node, &percpu_pool->free_objs); percpu_pool->obj_free++; local_irq_restore(flags); return; @@ -368,19 +364,19 @@ static void __free_object(struct debug_obj *obj) } =20 free_to_obj_pool: - raw_spin_lock(&pool_lock); + hlist_bl_lock(&obj_pool); work =3D (obj_pool_free > debug_objects_pool_size) && obj_cache && (obj_nr_tofree < ODEBUG_FREE_WORK_MAX); obj_pool_used--; =20 if (work) { WRITE_ONCE(obj_nr_tofree, obj_nr_tofree + 1); - hlist_add_head(&obj->node, &obj_to_free); + hlist_bl_add_head(&obj->node, &obj_to_free); if (lookahead_count) { WRITE_ONCE(obj_nr_tofree, obj_nr_tofree + lookahead_count); obj_pool_used -=3D lookahead_count; while (lookahead_count) { - hlist_add_head(&objs[--lookahead_count]->node, + hlist_bl_add_head(&objs[--lookahead_count]->node, &obj_to_free); } } @@ -394,24 +390,24 @@ static void __free_object(struct debug_obj *obj) */ for (i =3D 0; i < ODEBUG_BATCH_SIZE; i++) { obj =3D __alloc_object(&obj_pool); - hlist_add_head(&obj->node, &obj_to_free); + hlist_bl_add_head(&obj->node, &obj_to_free); WRITE_ONCE(obj_pool_free, obj_pool_free - 1); WRITE_ONCE(obj_nr_tofree, obj_nr_tofree + 1); } } } else { WRITE_ONCE(obj_pool_free, obj_pool_free + 1); - hlist_add_head(&obj->node, &obj_pool); + hlist_bl_add_head(&obj->node, &obj_pool); if (lookahead_count) { WRITE_ONCE(obj_pool_free, obj_pool_free + lookahead_count); obj_pool_used -=3D lookahead_count; while (lookahead_count) { - hlist_add_head(&objs[--lookahead_count]->node, + hlist_bl_add_head(&objs[--lookahead_count]->node, &obj_pool); } } } - raw_spin_unlock(&pool_lock); + hlist_bl_unlock(&obj_pool); local_irq_restore(flags); } =20 @@ -432,21 +428,21 @@ static void free_object(struct debug_obj *obj) static int object_cpu_offline(unsigned int cpu) { struct debug_percpu_free *percpu_pool; - struct hlist_node *tmp; + struct hlist_bl_node *tmp, *n; struct debug_obj *obj; unsigned long flags; =20 /* Remote access is safe as the CPU is dead already */ percpu_pool =3D per_cpu_ptr(&percpu_obj_pool, cpu); - hlist_for_each_entry_safe(obj, tmp, &percpu_pool->free_objs, node) { - hlist_del(&obj->node); + hlist_bl_for_each_entry_safe(obj, tmp, n, &percpu_pool->free_objs, node) { + hlist_bl_del(&obj->node); kmem_cache_free(obj_cache, obj); } =20 - raw_spin_lock_irqsave(&pool_lock, flags); + hlist_bl_lock_irqsave(&obj_pool, flags); obj_pool_used -=3D percpu_pool->obj_free; debug_objects_freed +=3D percpu_pool->obj_free; - raw_spin_unlock_irqrestore(&pool_lock, flags); + hlist_bl_unlock_irqrestore(&obj_pool, flags); =20 percpu_pool->obj_free =3D 0; =20 @@ -460,23 +456,23 @@ static int object_cpu_offline(unsigned int cpu) */ static void debug_objects_oom(void) { - struct debug_bucket *db =3D obj_hash; - struct hlist_node *tmp; - HLIST_HEAD(freelist); + struct hlist_bl_head *list =3D obj_hash; + struct hlist_bl_node *tmp; + HLIST_BL_HEAD(freelist); struct debug_obj *obj; unsigned long flags; int i; =20 pr_warn("Out of memory. ODEBUG disabled\n"); =20 - for (i =3D 0; i < ODEBUG_HASH_SIZE; i++, db++) { - raw_spin_lock_irqsave(&db->lock, flags); - hlist_move_list(&db->list, &freelist); - raw_spin_unlock_irqrestore(&db->lock, flags); + for (i =3D 0; i < ODEBUG_HASH_SIZE; i++, list++) { + hlist_bl_lock_irqsave(list, flags); + hlist_bl_move_list(list, &freelist); + hlist_bl_unlock_irqrestore(list, flags); =20 /* Now free them */ hlist_for_each_entry_safe(obj, tmp, &freelist, node) { - hlist_del(&obj->node); + hlist_bl_del(&obj->node); free_object(obj); } } @@ -486,7 +482,7 @@ static void debug_objects_oom(void) * We use the pfn of the address for the hash. That way we can check * for freed objects simply by checking the affected bucket. */ -static struct debug_bucket *get_bucket(unsigned long addr) +static struct hlist_bl_head *get_bucket(unsigned long addr) { unsigned long hash; =20 @@ -558,11 +554,11 @@ static void debug_object_is_on_stack(void *addr, int = onstack) WARN_ON(1); } =20 -static struct debug_obj *lookup_object_or_alloc(void *addr, struct debug_b= ucket *b, +static struct debug_obj *lookup_object_or_alloc(void *addr, struct hlist_b= l_head *list, const struct debug_obj_descr *descr, bool onstack, bool alloc_ifstatic) { - struct debug_obj *obj =3D lookup_object(addr, b); + struct debug_obj *obj =3D lookup_object(addr, list); enum debug_obj_state state =3D ODEBUG_STATE_NONE; =20 if (likely(obj)) @@ -585,7 +581,7 @@ static struct debug_obj *lookup_object_or_alloc(void *a= ddr, struct debug_bucket state =3D ODEBUG_STATE_INIT; } =20 - obj =3D alloc_object(addr, b, descr); + obj =3D alloc_object(addr, list, descr); if (likely(obj)) { obj->state =3D state; debug_object_is_on_stack(addr, onstack); @@ -622,18 +618,18 @@ static void __debug_object_init(void *addr, const struct debug_obj_descr *descr, int o= nstack) { struct debug_obj *obj, o; - struct debug_bucket *db; + struct hlist_bl_head *list; unsigned long flags; =20 debug_objects_fill_pool(); =20 - db =3D get_bucket((unsigned long) addr); + list =3D get_bucket((unsigned long) addr); =20 - raw_spin_lock_irqsave(&db->lock, flags); + hlist_bl_lock_irqsave(list, flags); =20 - obj =3D lookup_object_or_alloc(addr, db, descr, onstack, false); + obj =3D lookup_object_or_alloc(addr, list, descr, onstack, false); if (unlikely(!obj)) { - raw_spin_unlock_irqrestore(&db->lock, flags); + hlist_bl_unlock_irqrestore(list, flags); debug_objects_oom(); return; } @@ -643,14 +639,14 @@ __debug_object_init(void *addr, const struct debug_ob= j_descr *descr, int onstack case ODEBUG_STATE_INIT: case ODEBUG_STATE_INACTIVE: obj->state =3D ODEBUG_STATE_INIT; - raw_spin_unlock_irqrestore(&db->lock, flags); + hlist_bl_unlock_irqrestore(list, flags); return; default: break; } =20 o =3D *obj; - raw_spin_unlock_irqrestore(&db->lock, flags); + hlist_bl_unlock_irqrestore(list, flags); debug_print_object(&o, "init"); =20 if (o.state =3D=3D ODEBUG_STATE_ACTIVE) @@ -695,7 +691,7 @@ EXPORT_SYMBOL_GPL(debug_object_init_on_stack); int debug_object_activate(void *addr, const struct debug_obj_descr *descr) { struct debug_obj o =3D { .object =3D addr, .state =3D ODEBUG_STATE_NOTAVA= ILABLE, .descr =3D descr }; - struct debug_bucket *db; + struct hlist_bl_head *list; struct debug_obj *obj; unsigned long flags; =20 @@ -704,13 +700,13 @@ int debug_object_activate(void *addr, const struct de= bug_obj_descr *descr) =20 debug_objects_fill_pool(); =20 - db =3D get_bucket((unsigned long) addr); + list =3D get_bucket((unsigned long) addr); =20 - raw_spin_lock_irqsave(&db->lock, flags); + hlist_bl_lock_irqsave(list, flags); =20 - obj =3D lookup_object_or_alloc(addr, db, descr, false, true); + obj =3D lookup_object_or_alloc(addr, list, descr, false, true); if (unlikely(!obj)) { - raw_spin_unlock_irqrestore(&db->lock, flags); + hlist_bl_unlock_irqrestore(list, flags); debug_objects_oom(); return 0; } else if (likely(!IS_ERR(obj))) { @@ -724,12 +720,12 @@ int debug_object_activate(void *addr, const struct de= bug_obj_descr *descr) obj->state =3D ODEBUG_STATE_ACTIVE; fallthrough; default: - raw_spin_unlock_irqrestore(&db->lock, flags); + hlist_bl_unlock_irqrestore(list, flags); return 0; } } =20 - raw_spin_unlock_irqrestore(&db->lock, flags); + hlist_bl_unlock_irqrestore(list, flags); debug_print_object(&o, "activate"); =20 switch (o.state) { @@ -752,18 +748,18 @@ EXPORT_SYMBOL_GPL(debug_object_activate); void debug_object_deactivate(void *addr, const struct debug_obj_descr *des= cr) { struct debug_obj o =3D { .object =3D addr, .state =3D ODEBUG_STATE_NOTAVA= ILABLE, .descr =3D descr }; - struct debug_bucket *db; + struct hlist_bl_head *list; struct debug_obj *obj; unsigned long flags; =20 if (!debug_objects_enabled) return; =20 - db =3D get_bucket((unsigned long) addr); + list =3D get_bucket((unsigned long) addr); =20 - raw_spin_lock_irqsave(&db->lock, flags); + hlist_bl_lock_irqsave(list, flags); =20 - obj =3D lookup_object(addr, db); + obj =3D lookup_object(addr, list); if (obj) { switch (obj->state) { case ODEBUG_STATE_DESTROYED: @@ -776,13 +772,13 @@ void debug_object_deactivate(void *addr, const struct= debug_obj_descr *descr) obj->state =3D ODEBUG_STATE_INACTIVE; fallthrough; default: - raw_spin_unlock_irqrestore(&db->lock, flags); + hlist_bl_unlock_irqrestore(list, flags); return; } o =3D *obj; } =20 - raw_spin_unlock_irqrestore(&db->lock, flags); + hlist_bl_unlock_irqrestore(list, flags); debug_print_object(&o, "deactivate"); } EXPORT_SYMBOL_GPL(debug_object_deactivate); @@ -795,19 +791,19 @@ EXPORT_SYMBOL_GPL(debug_object_deactivate); void debug_object_destroy(void *addr, const struct debug_obj_descr *descr) { struct debug_obj *obj, o; - struct debug_bucket *db; + struct hlist_bl_head *list; unsigned long flags; =20 if (!debug_objects_enabled) return; =20 - db =3D get_bucket((unsigned long) addr); + list =3D get_bucket((unsigned long) addr); =20 - raw_spin_lock_irqsave(&db->lock, flags); + hlist_bl_lock_irqsave(list, flags); =20 - obj =3D lookup_object(addr, db); + obj =3D lookup_object(addr, list); if (!obj) { - raw_spin_unlock_irqrestore(&db->lock, flags); + hlist_bl_unlock_irqrestore(list, flags); return; } =20 @@ -821,12 +817,12 @@ void debug_object_destroy(void *addr, const struct de= bug_obj_descr *descr) obj->state =3D ODEBUG_STATE_DESTROYED; fallthrough; default: - raw_spin_unlock_irqrestore(&db->lock, flags); + hlist_bl_unlock_irqrestore(list, flags); return; } =20 o =3D *obj; - raw_spin_unlock_irqrestore(&db->lock, flags); + hlist_bl_unlock_irqrestore(list, flags); debug_print_object(&o, "destroy"); =20 if (o.state =3D=3D ODEBUG_STATE_ACTIVE) @@ -842,19 +838,19 @@ EXPORT_SYMBOL_GPL(debug_object_destroy); void debug_object_free(void *addr, const struct debug_obj_descr *descr) { struct debug_obj *obj, o; - struct debug_bucket *db; + struct hlist_bl_head *list; unsigned long flags; =20 if (!debug_objects_enabled) return; =20 - db =3D get_bucket((unsigned long) addr); + list =3D get_bucket((unsigned long) addr); =20 - raw_spin_lock_irqsave(&db->lock, flags); + hlist_bl_lock_irqsave(list, flags); =20 - obj =3D lookup_object(addr, db); + obj =3D lookup_object(addr, list); if (!obj) { - raw_spin_unlock_irqrestore(&db->lock, flags); + hlist_bl_unlock_irqrestore(list, flags); return; } =20 @@ -862,14 +858,14 @@ void debug_object_free(void *addr, const struct debug= _obj_descr *descr) case ODEBUG_STATE_ACTIVE: break; default: - hlist_del(&obj->node); - raw_spin_unlock_irqrestore(&db->lock, flags); + hlist_bl_del(&obj->node); + hlist_bl_unlock_irqrestore(list, flags); free_object(obj); return; } =20 o =3D *obj; - raw_spin_unlock_irqrestore(&db->lock, flags); + hlist_bl_unlock_irqrestore(list, flags); debug_print_object(&o, "free"); =20 debug_object_fixup(descr->fixup_free, addr, o.state); @@ -884,7 +880,7 @@ EXPORT_SYMBOL_GPL(debug_object_free); void debug_object_assert_init(void *addr, const struct debug_obj_descr *de= scr) { struct debug_obj o =3D { .object =3D addr, .state =3D ODEBUG_STATE_NOTAVA= ILABLE, .descr =3D descr }; - struct debug_bucket *db; + struct hlist_bl_head *list; struct debug_obj *obj; unsigned long flags; =20 @@ -893,11 +889,11 @@ void debug_object_assert_init(void *addr, const struc= t debug_obj_descr *descr) =20 debug_objects_fill_pool(); =20 - db =3D get_bucket((unsigned long) addr); + list =3D get_bucket((unsigned long) addr); =20 - raw_spin_lock_irqsave(&db->lock, flags); - obj =3D lookup_object_or_alloc(addr, db, descr, false, true); - raw_spin_unlock_irqrestore(&db->lock, flags); + hlist_bl_lock_irqsave(list, flags); + obj =3D lookup_object_or_alloc(addr, list, descr, false, true); + hlist_bl_unlock_irqrestore(list, flags); if (likely(!IS_ERR_OR_NULL(obj))) return; =20 @@ -925,25 +921,25 @@ debug_object_active_state(void *addr, const struct de= bug_obj_descr *descr, unsigned int expect, unsigned int next) { struct debug_obj o =3D { .object =3D addr, .state =3D ODEBUG_STATE_NOTAVA= ILABLE, .descr =3D descr }; - struct debug_bucket *db; + struct hlist_bl_head *list; struct debug_obj *obj; unsigned long flags; =20 if (!debug_objects_enabled) return; =20 - db =3D get_bucket((unsigned long) addr); + list =3D get_bucket((unsigned long) addr); =20 - raw_spin_lock_irqsave(&db->lock, flags); + hlist_bl_lock_irqsave(list, flags); =20 - obj =3D lookup_object(addr, db); + obj =3D lookup_object(addr, list); if (obj) { switch (obj->state) { case ODEBUG_STATE_ACTIVE: if (obj->astate !=3D expect) break; obj->astate =3D next; - raw_spin_unlock_irqrestore(&db->lock, flags); + hlist_bl_unlock_irqrestore(list, flags); return; default: break; @@ -951,7 +947,7 @@ debug_object_active_state(void *addr, const struct debu= g_obj_descr *descr, o =3D *obj; } =20 - raw_spin_unlock_irqrestore(&db->lock, flags); + hlist_bl_unlock_irqrestore(list, flags); debug_print_object(&o, "active_state"); } EXPORT_SYMBOL_GPL(debug_object_active_state); @@ -962,8 +958,8 @@ static void __debug_check_no_obj_freed(const void *addr= ess, unsigned long size) unsigned long flags, oaddr, saddr, eaddr, paddr, chunks; int cnt, objs_checked =3D 0; struct debug_obj *obj, o; - struct debug_bucket *db; - struct hlist_node *tmp; + struct hlist_bl_head *list; + struct hlist_bl_node *tmp, *n; =20 saddr =3D (unsigned long) address; eaddr =3D saddr + size; @@ -972,12 +968,12 @@ static void __debug_check_no_obj_freed(const void *ad= dress, unsigned long size) chunks >>=3D ODEBUG_CHUNK_SHIFT; =20 for (;chunks > 0; chunks--, paddr +=3D ODEBUG_CHUNK_SIZE) { - db =3D get_bucket(paddr); + list =3D get_bucket(paddr); =20 repeat: cnt =3D 0; - raw_spin_lock_irqsave(&db->lock, flags); - hlist_for_each_entry_safe(obj, tmp, &db->list, node) { + hlist_bl_lock_irqsave(list, flags); + hlist_bl_for_each_entry_safe(obj, tmp, n, list, node) { cnt++; oaddr =3D (unsigned long) obj->object; if (oaddr < saddr || oaddr >=3D eaddr) @@ -986,17 +982,17 @@ static void __debug_check_no_obj_freed(const void *ad= dress, unsigned long size) switch (obj->state) { case ODEBUG_STATE_ACTIVE: o =3D *obj; - raw_spin_unlock_irqrestore(&db->lock, flags); + hlist_bl_unlock_irqrestore(list, flags); debug_print_object(&o, "free"); debug_object_fixup(o.descr->fixup_free, (void *)oaddr, o.state); goto repeat; default: - hlist_del(&obj->node); + hlist_bl_del(&obj->node); __free_object(obj); break; } } - raw_spin_unlock_irqrestore(&db->lock, flags); + hlist_bl_unlock_irqrestore(list, flags); =20 if (cnt > debug_objects_maxchain) debug_objects_maxchain =3D cnt; @@ -1162,16 +1158,16 @@ static bool __init fixup_free(void *addr, enum debu= g_obj_state state) static int __init check_results(void *addr, enum debug_obj_state state, int fixups, int warn= ings) { - struct debug_bucket *db; + struct hlist_bl_head *list; struct debug_obj *obj; unsigned long flags; int res =3D -EINVAL; =20 - db =3D get_bucket((unsigned long) addr); + list =3D get_bucket((unsigned long) addr); =20 - raw_spin_lock_irqsave(&db->lock, flags); + hlist_bl_lock_irqsave(list, flags); =20 - obj =3D lookup_object(addr, db); + obj =3D lookup_object(addr, list); if (!obj && state !=3D ODEBUG_STATE_NONE) { WARN(1, KERN_ERR "ODEBUG: selftest object not found\n"); goto out; @@ -1193,7 +1189,7 @@ check_results(void *addr, enum debug_obj_state state,= int fixups, int warnings) } res =3D 0; out: - raw_spin_unlock_irqrestore(&db->lock, flags); + hlist_bl_unlock_irqrestore(list, flags); if (res) debug_objects_enabled =3D 0; return res; @@ -1294,10 +1290,10 @@ void __init debug_objects_early_init(void) int i; =20 for (i =3D 0; i < ODEBUG_HASH_SIZE; i++) - raw_spin_lock_init(&obj_hash[i].lock); + INIT_HLIST_BL_HEAD(&obj_hash[i]); =20 for (i =3D 0; i < ODEBUG_POOL_SIZE; i++) - hlist_add_head(&obj_static_pool[i].node, &obj_pool); + hlist_bl_add_head(&obj_static_pool[i].node, &obj_pool); } =20 /* @@ -1305,17 +1301,17 @@ void __init debug_objects_early_init(void) */ static int __init debug_objects_replace_static_objects(void) { - struct debug_bucket *db =3D obj_hash; - struct hlist_node *tmp; + struct hlist_bl_head *head =3D obj_hash; + struct hlist_bl_node *tmp, *n; struct debug_obj *obj, *new; - HLIST_HEAD(objects); + HLIST_BL_HEAD(objects); int i, cnt =3D 0; =20 for (i =3D 0; i < ODEBUG_POOL_SIZE; i++) { obj =3D kmem_cache_zalloc(obj_cache, GFP_KERNEL); if (!obj) goto free; - hlist_add_head(&obj->node, &objects); + hlist_bl_add_head(&obj->node, &objects); } =20 debug_objects_allocated +=3D i; @@ -1327,21 +1323,21 @@ static int __init debug_objects_replace_static_obje= cts(void) */ =20 /* Remove the statically allocated objects from the pool */ - hlist_for_each_entry_safe(obj, tmp, &obj_pool, node) - hlist_del(&obj->node); + hlist_bl_for_each_entry_safe(obj, tmp, n, &obj_pool, node) + hlist_bl_del(&obj->node); /* Move the allocated objects to the pool */ - hlist_move_list(&objects, &obj_pool); + hlist_bl_move_list(&objects, &obj_pool); =20 /* Replace the active object references */ - for (i =3D 0; i < ODEBUG_HASH_SIZE; i++, db++) { - hlist_move_list(&db->list, &objects); + for (i =3D 0; i < ODEBUG_HASH_SIZE; i++, head++) { + hlist_bl_move_list(head, &objects); =20 - hlist_for_each_entry(obj, &objects, node) { - new =3D hlist_entry(obj_pool.first, typeof(*obj), node); - hlist_del(&new->node); + hlist_bl_for_each_entry(obj, tmp, &objects, node) { + new =3D hlist_bl_entry(hlist_bl_first(&obj_pool), typeof(*obj), node); + hlist_bl_del(&new->node); /* copy object data */ *new =3D *obj; - hlist_add_head(&new->node, &db->list); + hlist_bl_add_head(&new->node, head); cnt++; } } @@ -1350,8 +1346,8 @@ static int __init debug_objects_replace_static_object= s(void) cnt, obj_pool_used); return 0; free: - hlist_for_each_entry_safe(obj, tmp, &objects, node) { - hlist_del(&obj->node); + hlist_bl_for_each_entry_safe(obj, tmp, n, &objects, node) { + hlist_bl_del(&obj->node); kmem_cache_free(obj_cache, obj); } return -ENOMEM; @@ -1377,7 +1373,7 @@ void __init debug_objects_mem_init(void) * completeness. */ for_each_possible_cpu(cpu) - INIT_HLIST_HEAD(&per_cpu(percpu_obj_pool.free_objs, cpu)); + INIT_HLIST_BL_HEAD(&per_cpu(percpu_obj_pool.free_objs, cpu)); =20 obj_cache =3D kmem_cache_create("debug_objects_cache", sizeof (struct debug_obj), 0, --=20 2.45.2