From nobody Fri Dec 19 15:51:27 2025 Received: from invmail4.hynix.com (exvmail4.hynix.com [166.125.252.92]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 2E63A4B1E64; Mon, 19 May 2025 09:18:44 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=166.125.252.92 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747646330; cv=none; b=U+L744D367UX74egPfI5KIwEeptV/pyf8j/rsY81yWherXntna+IWPE5N3cICKTMod+biCPCDQhB6NzIy9avLD4/za0pfvYQFLMdSsVXopAD4XxpbvC4RmbWGMK/64DZHGTXt9sZN+K6wgAW+uSsU7UrusfeNelsQyuV1GMNmO0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747646330; c=relaxed/simple; bh=E7HG0KbyHora1Jy7kN4bmYK0Ru8v8roLf/CFToUPmtA=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References; b=jORNHJZ5cWTldXXpnWQhm6IfOGOrFbI9PxCGKbvaq5nsfWAvAua1CiELxVo1eX91gE0CPC+HmFv2vrB6qaDIiG6nM94GZPM4cd3fNCSFPVm7nVswHVkU/AfLwxlLmE2C1SDDuXhQB3+98mXjbZPwZKr+SOrVSz8lWBKS4Vpr2mU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com; spf=pass smtp.mailfrom=sk.com; arc=none smtp.client-ip=166.125.252.92 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=sk.com X-AuditID: a67dfc5b-669ff7000002311f-4d-682af76c7dad From: Byungchul Park To: linux-kernel@vger.kernel.org Cc: kernel_team@skhynix.com, torvalds@linux-foundation.org, damien.lemoal@opensource.wdc.com, linux-ide@vger.kernel.org, adilger.kernel@dilger.ca, linux-ext4@vger.kernel.org, mingo@redhat.com, peterz@infradead.org, will@kernel.org, tglx@linutronix.de, rostedt@goodmis.org, joel@joelfernandes.org, sashal@kernel.org, daniel.vetter@ffwll.ch, duyuyang@gmail.com, johannes.berg@intel.com, tj@kernel.org, tytso@mit.edu, willy@infradead.org, david@fromorbit.com, amir73il@gmail.com, gregkh@linuxfoundation.org, kernel-team@lge.com, linux-mm@kvack.org, akpm@linux-foundation.org, mhocko@kernel.org, minchan@kernel.org, hannes@cmpxchg.org, vdavydov.dev@gmail.com, sj@kernel.org, jglisse@redhat.com, dennis@kernel.org, cl@linux.com, penberg@kernel.org, rientjes@google.com, vbabka@suse.cz, ngupta@vflare.org, linux-block@vger.kernel.org, josef@toxicpanda.com, linux-fsdevel@vger.kernel.org, jack@suse.cz, jlayton@kernel.org, dan.j.williams@intel.com, hch@infradead.org, djwong@kernel.org, dri-devel@lists.freedesktop.org, rodrigosiqueiramelo@gmail.com, melissa.srw@gmail.com, hamohammed.sa@gmail.com, harry.yoo@oracle.com, chris.p.wilson@intel.com, gwan-gyeong.mun@intel.com, max.byungchul.park@gmail.com, boqun.feng@gmail.com, longman@redhat.com, yskelg@gmail.com, yunseong.kim@ericsson.com, yeoreum.yun@arm.com, netdev@vger.kernel.org, matthew.brost@intel.com, her0gyugyu@gmail.com Subject: [PATCH v16 01/42] llist: move llist_{head,node} definition to types.h Date: Mon, 19 May 2025 18:17:45 +0900 Message-Id: <20250519091826.19752-2-byungchul@sk.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20250519091826.19752-1-byungchul@sk.com> References: <20250519091826.19752-1-byungchul@sk.com> X-Brightmail-Tracker: H4sIAAAAAAAAAzWSbUxScRSH+997ufdC0W70dtW2is1ebL1Y2k5btVab3bVe17f6UJS3YCE6 VMy2Ni00i8BsqUvNgBwxwTJoRSVlOk0zy4qQzKysWYRms6AsegFbX86e7fx+z/lyaFzSJoil FaosXq2SKaWkiBANTTAvVH5LkC954V8Jwa9FBFRdtpPQdcmGwH41HwN/y3roDg0i+Nn5CIfy 0i4Epjcvcbja2ofAbT1CwtN3E8ETHCahvVRHwtELl0l4HAhj0Ft2GgObYxO8sgwQ0HHKjEG5 n4TK8qNYZHzAYNRSS4ElLx76rRUUhN8kQnufVwDungVwtrqXhAZ3OwGtrn4Mnt6sIqHP/kcA Ha1tBIQMcdBVohdA3SczCYGQBQdLcJiCJ41GDFqN06BeGxEWfvktgHv6RgwKa65g4Hl+C8Ht otcYOOxeEpqDgxg4HaU4/LjYgqDfMERBwclRCirzDQh0BWUEaHuT4ef3yOVzXxMh/3w9AXW/ vGjNKs5ebUdc8+AwzmmdOdyP4DOSc4eMBHffzHI3Kl5SnPZ2D8UZHdmc05rAXWjwY5xpJCjg HLXHSc4xcpriTgx5MO7Tw4fU1hk7RCtTeaVCw6sXr94tkpf6TVSGSXSwzv2NyEN36BNISLNM Ens9cFHwn1uKAyjKJDOX9flG8ShPYWaxTv1AJCOiccY7nu0+93wsNJnZzHY0FGFRJph41tUd JKMsZpLZzyW3qH/SmaytvnFMJGSWsz265rGuJJLx2KqJqJRlKoVsWWMF+a8Qw961+ohTSGxE 42qRRKHSpMkUyqRF8lyV4uCivelpDhT5L8vh8E4XGuna3oQYGkkniOvd8+USgUyTmZvWhFga l04R1zrnySXiVFnuIV6dvkudreQzm1AcTUini5eGclIlzH5ZFn+A5zN49f8tRgtj89AZV3tW /4vpiscdccc2zP6u01G+mBWdse+kb7N83Qr9vpsPijU2y+JA53D6umTPnDzzrBjD+vA2y4Dw 2h/XpKkj3t05NVpNSlKK7n1KmpW3J6W3ZOhly44UqsJ7txRrD21cRhFtO7Lvrxus2VNS9tG3 Kd67r6qnIHFjQrZhrdu0UEpkymWJCbg6U/YXs+vX6FsDAAA= X-Brightmail-Tracker: H4sIAAAAAAAAAzWSfUiTaxiHe57309XibUm9GFGMpDJOJ6Wdc0MRfVA9FEkIFQSRM1/cappn M08GlaaGaVoWJpraNFmyzdLNQk+uZDuullR23NEUtaN0pKVpH25p2cc0+ufmgvt3X/c/P55S mJkwXpuUIumT1DolK6Nl0esyf9F9jNCsyfMsA/94Dg1lt6wstN+0ILA2ZGDwtW6HrsAIgs+P n1JQXNSOoHKgj4IGdz8CR80ZFjpezgWvf4wFT1EeC5nXb7HwbHgKQ++VSxgstl3wwjREQ9vF KgzFPhauFmfi4HiFYdJk5sCUHg6DNaUcTA1Egqe/kwFXuYcBR88qKKnoZaHZ4aHB3TiIoeOv Mhb6rd8YaHM/pCFQsAjaC/MZqB2tYmE4YKLA5B/j4J8WIwa3cQHUZQWtZz98ZeBBfguGs9X1 GLzddxHcy/kPg83ayYLLP4LBbiui4NONVgSDBW84yD4/ycHVjAIEedlXaMjqVcHnieDn8vFI yLhWR0Ptl060cQOxVlgRcY2MUSTL/if55P+XJY6AkSaPqkTSVNrHkax7PRwx2o4Re00Eud7s w6TyvZ8hNvM5ltjeX+JI7hsvJqNPnnC7F++XrY+XdNpUSf/rhliZpshXySVXyo7XOj7S6eg+ n4tCeFFYK7ZeGEbTzArLxefPJ6lpDhWWivb8ISYXyXhK6JwtdpV3z4TmC9FiW3MOnmZaCBcb u/zsNMsFlfi28C73Q7pEtNS1zIhChN/EnjzXzK0imPFaKuiLSGZEs8woVJuUmqjW6lSrDUc0 aUna46sPHU20oWCDTCenChvReMd2JxJ4pJwjr3Os1CgYdaohLdGJRJ5ShsrN9hUahTxenXZC 0h89qD+mkwxOtIinlQvlO/ZJsQohQZ0iHZGkZEn/c4v5kLB0NCBEhTe/vr/573bekvJ7yeje +iFqVxhz2f8u9u3aQFv+xJ2vfywff70+Oq609zRJKHsUFYhZnNz0/wJV3NPInVF7Dlcl2Bi7 UOM81T2vryl8S4x3/7OtN5VOe5+wqXo4pCn1XI8bn769x8VNbJtXHSpm+5QH5k+WGNNUKQMH H74cVNIGjToygtIb1N8B2g1/pz0DAAA= X-CFilter-Loop: Reflected Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" llist_head and llist_node can be used by some other header files. For example, dept for tracking dependencies uses llist in its header. To avoid header dependency, move them to types.h. Signed-off-by: Byungchul Park --- include/linux/llist.h | 8 -------- include/linux/types.h | 8 ++++++++ 2 files changed, 8 insertions(+), 8 deletions(-) diff --git a/include/linux/llist.h b/include/linux/llist.h index 2c982ff7475a..3ac071857612 100644 --- a/include/linux/llist.h +++ b/include/linux/llist.h @@ -53,14 +53,6 @@ #include #include =20 -struct llist_head { - struct llist_node *first; -}; - -struct llist_node { - struct llist_node *next; -}; - #define LLIST_HEAD_INIT(name) { NULL } #define LLIST_HEAD(name) struct llist_head name =3D LLIST_HEAD_INIT(name) =20 diff --git a/include/linux/types.h b/include/linux/types.h index 49b79c8bb1a9..c727cc2249e8 100644 --- a/include/linux/types.h +++ b/include/linux/types.h @@ -204,6 +204,14 @@ struct hlist_node { struct hlist_node *next, **pprev; }; =20 +struct llist_head { + struct llist_node *first; +}; + +struct llist_node { + struct llist_node *next; +}; + struct ustat { __kernel_daddr_t f_tfree; #ifdef CONFIG_ARCH_32BIT_USTAT_F_TINODE --=20 2.17.1 From nobody Fri Dec 19 15:51:27 2025 Received: from invmail4.hynix.com (exvmail4.hynix.com [166.125.252.92]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 366CC26C399; Mon, 19 May 2025 09:18:44 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=166.125.252.92 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747646334; cv=none; b=joUZF2msxcILaj2VY3a0nEw7O3dElTsOjT9n/EcBNCORKaeoFsrZuSzcrFvMFXI08bP76fm3yLeS7g/pKic2LP9cTrHZwLTqMV96MVpxFXGYAxGFPuR0WUfFOi2sVWv/fTgDFHopJQDUnm4qTiR0/4GmGPOCUGcfCNiytwO4VHQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747646334; c=relaxed/simple; bh=LpRXz0duwc4dLoAx4URhdlf+u3eO/CuCSmsA+pRZdFc=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References; b=dYdwNB9up5o6JBSdKYIEgGmLQHMf3yR1r5dkCYFNZ6ND0uWrp8V0NguqAvFy32PNEz5Mnz/qxnKBLLHHeVo3WSPx/rMNaNq6gnFJvZ5zbP7czEx0Ff2x6baItD41125FnSe/gbF/9rEk6yRZW4zVRe46rVGnGxO0zDzIiN3CWWw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com; spf=pass smtp.mailfrom=sk.com; arc=none smtp.client-ip=166.125.252.92 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=sk.com X-AuditID: a67dfc5b-669ff7000002311f-5e-682af76ce22a From: Byungchul Park To: linux-kernel@vger.kernel.org Cc: kernel_team@skhynix.com, torvalds@linux-foundation.org, damien.lemoal@opensource.wdc.com, linux-ide@vger.kernel.org, adilger.kernel@dilger.ca, linux-ext4@vger.kernel.org, mingo@redhat.com, peterz@infradead.org, will@kernel.org, tglx@linutronix.de, rostedt@goodmis.org, joel@joelfernandes.org, sashal@kernel.org, daniel.vetter@ffwll.ch, duyuyang@gmail.com, johannes.berg@intel.com, tj@kernel.org, tytso@mit.edu, willy@infradead.org, david@fromorbit.com, amir73il@gmail.com, gregkh@linuxfoundation.org, kernel-team@lge.com, linux-mm@kvack.org, akpm@linux-foundation.org, mhocko@kernel.org, minchan@kernel.org, hannes@cmpxchg.org, vdavydov.dev@gmail.com, sj@kernel.org, jglisse@redhat.com, dennis@kernel.org, cl@linux.com, penberg@kernel.org, rientjes@google.com, vbabka@suse.cz, ngupta@vflare.org, linux-block@vger.kernel.org, josef@toxicpanda.com, linux-fsdevel@vger.kernel.org, jack@suse.cz, jlayton@kernel.org, dan.j.williams@intel.com, hch@infradead.org, djwong@kernel.org, dri-devel@lists.freedesktop.org, rodrigosiqueiramelo@gmail.com, melissa.srw@gmail.com, hamohammed.sa@gmail.com, harry.yoo@oracle.com, chris.p.wilson@intel.com, gwan-gyeong.mun@intel.com, max.byungchul.park@gmail.com, boqun.feng@gmail.com, longman@redhat.com, yskelg@gmail.com, yunseong.kim@ericsson.com, yeoreum.yun@arm.com, netdev@vger.kernel.org, matthew.brost@intel.com, her0gyugyu@gmail.com Subject: [PATCH v16 02/42] dept: implement DEPT(DEPendency Tracker) Date: Mon, 19 May 2025 18:17:46 +0900 Message-Id: <20250519091826.19752-3-byungchul@sk.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20250519091826.19752-1-byungchul@sk.com> References: <20250519091826.19752-1-byungchul@sk.com> X-Brightmail-Tracker: H4sIAAAAAAAAAzWSe0yTZxTG977ftdUuX6qZ32BmSzOn4kQxoCeZGuOy7M2yJSb+oZmX2dgv tgKFtIKyRIUBKlQYMwHGRfyApUNaB7aYKFrDaKhWBqKWe2ETDZFIbURbx2XMUvWfkyfnOc/v +efwlNrLxPAG4xHJZNSmaFglrQwsrl1rfBWnX//oPgOhl2doqG6ys9Dzhw2BvSUHw0TH19Af nkQw23WXgvLSHgS1D0coaPGMInA1/MTCg8fvgy8UZMFbamEht76JhXtP5zD4y85hsDm+g7+t 4zR0ltRhKJ9goao8F0fGEwzT1kYOrNkrYKyhkoO5hwngHe1jwDW0Bipq/CzccHlp8Fwdw/Cg tZqFUfv/DHR6btMQLo6Fnl+KGLj0rI6Fp2ErBdZQkIP7bTIGj/wBNOdFgKdezDNwq6gNw6nf LmPwDV5HcPPMPxgc9j4W3KFJDE5HKQUzv3cgGCsOcJB/dpqDqpxiBJb8Mhry/Ekw+2+k+fzL BMi50EzDpf/60LYtxF5jR8Q9GaRInvMomQn1ssQVlmlyp04k1ypHOJJ3c4gjsiODOBviSP2N CUxqp0IMcTQWsMQxdY4jhQEfJs+6u7kdH32v3KyTUgyZkmnd1gNKvV9env4omz3WdrEVZ6O7 vXQhUvCikCiWzM5R7/SVqk68oFlhpTgwMB3dLxU+EZ1F40whUvKU0LdI7D8/iBaMJcKXYkvB fBRECyvEisHTUa0SksT2C563BR+Ltua2KEghbBSHLO5oVh258dlq6AWoKNQrxMGuEH4T+FD8 s2GALkEqGb3XiNQGY2aq1pCSGK/PMhqOxR9MS3WgyINZj8/tuYqmena2I4FHmsWqZtdqvZrR ZpqzUtuRyFOapapG5yq9WqXTZv0omdJ+MGWkSOZ2FMvTmmWqDeGjOrVwSHtESpakdMn0zsW8 IiYbxVhWH/7ibM6mk5+m5e5WPne6O1ZJypnYGf9F88jwqyUFl/FwRUbvz18FF83Gy3+dTv78 MP3Nfl/X/h230b4T3Xfmg5S3sn482afYQtI/623auMumo/WdyyxSYDj/W2E5f+VxULnpVzlc qUMBdneis2a722Zv3bu5urVfToLyMg1t1msT4iiTWfsa49Ml+lwDAAA= X-Brightmail-Tracker: H4sIAAAAAAAAAzWSe0xTdxTH/f3uqxRrbiqbNyJRa4zKMhnTmjMfRNGEq1GjRjHRZHojd2tj QdJCFQyT2ooKUpUECCJQi6lAi3Qtm88aBrEbI2o3EKRBRHxEQoVEaBXBRyH6z8kn+Z7v5/xz JITcTs2WqNMyRG2aoFHQUlK6dZXxe83bWNUP1YEfITh6ioSLDQ4afFftCByNBgwDd5OgKxRA MH7vAQGlxT4El54+JqDR24vAU3OchvbnM6AjOExDa3EBDcbqBhr+G5zA0FNShMHu2gJPbC9J aDtnxVA6QEN5qRGHxysMY7Y6Bmy5C6G/5gIDE0/jobW3k4KWilYKPP7voKyyh4bbnlYSvNf7 MbTfvEhDr+MTBW3ef0gImaPBd76QgvohKw2DIRsBtuAwA/83WTB4Ld+C0xS25o18pODvwiYM eZd/x9DRfQvBnVN9GFyOThpaggEMblcxAe+v3EXQb37NwIkzYwyUG8wICk6UkGDqUcL4u/Dl itF4MFQ5Saj/0InWJvCOSgfiWwLDBG9yH+bfBx/SvCdkIfl/rRx/48Jjhjfd8TO8xZXJu2ti +erbA5i/9CZI8a660zTvelPE8PmvOzA/dP8+sy1mj3R1iqhR60VtXMJ+qarHEpP+LJc+0lR7 E+eiBw/JfBQh4djl3B/lbXiSaXYR9+jRGDHJUew8zl34kspHUgnBdkZyXRXdaDKYya7nGk9/ nCqT7EKurPvkFMtYJddc5f0incvZnU1Togh2BecvaJnqysM7HfZK8hySWtC0OhSlTtOnCmqN cqnuoCorTX1k6YFDqS4UfiFbzsT562i0PakZsRKkmC5zepao5JSg12WlNiNOQiiiZHXuxSq5 LEXIyha1h/ZpMzWirhlFS0jFLNmm3eJ+OfurkCEeFMV0Ufs1xZKI2bnIurtvxLp+r3m7YY4p 46e4317p05XujU92fLOoIYZafVUVXb/vT3PMoKakyF/7187AtcTxhL1rbMkj2UpFkmAY27jq yqj+RfLmo70bjr5NXLzsrP+e+Vpe38/O7AW+dSmhnGOWXa6ud5nzz2ZdXukRlL8kJ9bm+CKN kTnGG+qqHSuH4hSkTiXExxJanfAZ6O5XeT4DAAA= X-CFilter-Loop: Reflected Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" CURRENT STATUS Tested-by: Gwan-gyeong Mun Tested-by: Hyeonggon Yoo <42.hyeyoo@gmail.com> Tested-by: Yeoreum Yun Tested-by: Yunseong Kim -------------- lockdep tracks acquisition order of locks in order to detect deadlock, and IRQ and IRQ enable/disable state as well to take accident acquisitions into account. lockdep should be turned off once it detects and reports a deadlock since the data structure and algorithm are not reusable after detection because of the complex design. PROBLEM ------- *Waits* and their *events* that never reach eventually cause deadlock. However, lockdep is only interested in lock acquisition order, forcing to emulate lock acqusition even for just waits and events that have nothing to do with real lock. Even worse, no one likes lockdep's false positive detection because that prevents further one that might be more valuable. That's why all the kernel developers are sensitive to lockdep's false positive. Besides those, by tracking acquisition order, it cannot correctly deal with read lock and cross-event e.g. wait_for_completion()/complete() for deadlock detection. lockdep is no longer a good tool for that purpose. SOLUTION -------- Again, *waits* and their *events* that never reach eventually cause deadlock. The new solution, DEPT(DEPendency Tracker), focuses on waits and events themselves. dept tracks waits and events and report it if any event would be never reachable. dept does: . Works with read lock in the right way. . Works with any wait and event e.i. cross-event. . Continue to work even after reporting multiple times. . Provides simple and intuitive APIs. . Does exactly what dependency checker should do. Q & A ----- Q. Is this the first try ever to address the problem? A. No, cross-release feature (b09be676e0ff2 locking/lockdep: Implement the 'crossrelease' feature) addressed it that was a lockdep extension and merged but reverted shortly because: cross-release started to report valuable hidden problems but started to give report false positive reports as well. For sure, no one likes lockdep's false positive reports since it makes lockdep stop, preventing reporting further real problems. Q. Why not dept was developed as an extension of lockdep? A. lockdep definitely includes all the efforts great developers have made for a long time so as to be quite stable enough. But I had to design and implement newly because of the following: 1) lockdep was designed to track lock acquisition order. The APIs and implementation do not fit on wait-event model. 2) lockdep is turned off on detection including false positive. Which is terrible and prevents developing any extension for stronger detection. Q. Do you intend to totally replace lockdep? A. No, lockdep also checks if lock usage is correct. Of course, the dependency check routine should be replaced but the other functions should be still there. Q. Do you mean the dependency check routine should be replaced right away? A. No, I admit lockdep is stable enough thanks to great efforts kernel developers have made. lockdep and dept, both should be in the kernel until dept gets considered stable. Q. Stronger detection capability would give more false positive report. Which was a big problem when cross-release was introduced. Is it ok with dept? A. It's ok. dept allows multiple reporting thanks to simple and quite generalized design. Of course, false positive reports should be fixed anyway but it's no longer as a critical problem as it was. Signed-off-by: Byungchul Park Tested-by: Hyeonggon Yoo <42.hyeyoo@gmail.com> Tested-by: Gwan-gyeong Mun Tested-by: Yeoreum Yun Tested-by: Yunseong Kim --- include/linux/dept.h | 446 +++++ include/linux/hardirq.h | 3 + include/linux/sched.h | 110 +- init/init_task.c | 2 + init/main.c | 2 + kernel/Makefile | 1 + kernel/dependency/Makefile | 3 + kernel/dependency/dept.c | 3001 +++++++++++++++++++++++++++++++ kernel/dependency/dept_hash.h | 10 + kernel/dependency/dept_object.h | 13 + kernel/exit.c | 1 + kernel/fork.c | 2 + kernel/module/main.c | 4 + kernel/sched/core.c | 8 + lib/Kconfig.debug | 26 + lib/locking-selftest.c | 2 + 16 files changed, 3632 insertions(+), 2 deletions(-) create mode 100644 include/linux/dept.h create mode 100644 kernel/dependency/Makefile create mode 100644 kernel/dependency/dept.c create mode 100644 kernel/dependency/dept_hash.h create mode 100644 kernel/dependency/dept_object.h diff --git a/include/linux/dept.h b/include/linux/dept.h new file mode 100644 index 000000000000..5f0d2d8c8cbe --- /dev/null +++ b/include/linux/dept.h @@ -0,0 +1,446 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* + * DEPT(DEPendency Tracker) - runtime dependency tracker + * + * Started by Byungchul Park : + * + * Copyright (c) 2020 LG Electronics, Inc., Byungchul Park + * Copyright (c) 2024 SK hynix, Inc., Byungchul Park + */ + +#ifndef __LINUX_DEPT_H +#define __LINUX_DEPT_H + +#ifdef CONFIG_DEPT + +#include + +struct task_struct; + +#define DEPT_MAX_STACK_ENTRY 16 +#define DEPT_MAX_WAIT_HIST 64 +#define DEPT_MAX_ECXT_HELD 48 + +#define DEPT_MAX_SUBCLASSES 16 +#define DEPT_MAX_SUBCLASSES_EVT 2 +#define DEPT_MAX_SUBCLASSES_USR (DEPT_MAX_SUBCLASSES / DEPT_MAX_SUBCLASSE= S_EVT) +#define DEPT_MAX_SUBCLASSES_CACHE 2 + +#define DEPT_SIRQ 0 +#define DEPT_HIRQ 1 +#define DEPT_IRQS_NR 2 +#define DEPT_SIRQF (1UL << DEPT_SIRQ) +#define DEPT_HIRQF (1UL << DEPT_HIRQ) + +struct dept_ecxt; +struct dept_iecxt { + struct dept_ecxt *ecxt; + int enirq; + /* + * for preventing to add a new ecxt + */ + bool staled; +}; + +struct dept_wait; +struct dept_iwait { + struct dept_wait *wait; + int irq; + /* + * for preventing to add a new wait + */ + bool staled; + bool touched; +}; + +struct dept_class { + union { + struct llist_node pool_node; + struct { + /* + * reference counter for object management + */ + atomic_t ref; + + /* + * unique information about the class + */ + const char *name; + unsigned long key; + int sub_id; + + /* + * for BFS + */ + unsigned int bfs_gen; + struct dept_class *bfs_parent; + struct list_head bfs_node; + + /* + * for hashing this object + */ + struct hlist_node hash_node; + + /* + * for linking all classes + */ + struct list_head all_node; + + /* + * for associating its dependencies + */ + struct list_head dep_head; + struct list_head dep_rev_head; + + /* + * for tracking IRQ dependencies + */ + struct dept_iecxt iecxt[DEPT_IRQS_NR]; + struct dept_iwait iwait[DEPT_IRQS_NR]; + + /* + * classified by a map embedded in task_struct, + * not an explicit map + */ + bool sched_map; + }; + }; +}; + +struct dept_key { + union { + /* + * Each byte-wise address will be used as its key. + */ + char base[DEPT_MAX_SUBCLASSES]; + + /* + * for caching the main class pointer + */ + struct dept_class *classes[DEPT_MAX_SUBCLASSES_CACHE]; + }; +}; + +struct dept_map { + const char *name; + struct dept_key *keys; + + /* + * subclass that can be set from user + */ + int sub_u; + + /* + * It's local copy for fast access to the associated classes. + * Also used for dept_key for static maps. + */ + struct dept_key map_key; + + /* + * wait timestamp associated to this map + */ + unsigned int wgen; + + /* + * whether this map should be going to be checked or not + */ + bool nocheck; +}; + +#define DEPT_MAP_INITIALIZER(n, k) \ +{ \ + .name =3D #n, \ + .keys =3D (struct dept_key *)(k), \ + .sub_u =3D 0, \ + .map_key =3D { .classes =3D { NULL, } }, \ + .wgen =3D 0U, \ + .nocheck =3D false, \ +} + +struct dept_stack { + union { + struct llist_node pool_node; + struct { + /* + * reference counter for object management + */ + atomic_t ref; + + /* + * backtrace entries + */ + unsigned long raw[DEPT_MAX_STACK_ENTRY]; + int nr; + }; + }; +}; + +struct dept_ecxt { + union { + struct llist_node pool_node; + struct { + /* + * reference counter for object management + */ + atomic_t ref; + + /* + * function that entered to this ecxt + */ + const char *ecxt_fn; + + /* + * event function + */ + const char *event_fn; + + /* + * associated class + */ + struct dept_class *class; + + /* + * flag indicating which IRQ has been + * enabled within the event context + */ + unsigned long enirqf; + + /* + * where the IRQ-enabled happened + */ + unsigned long enirq_ip[DEPT_IRQS_NR]; + struct dept_stack *enirq_stack[DEPT_IRQS_NR]; + + /* + * where the event context started + */ + unsigned long ecxt_ip; + struct dept_stack *ecxt_stack; + + /* + * where the event triggered + */ + unsigned long event_ip; + struct dept_stack *event_stack; + }; + }; +}; + +struct dept_wait { + union { + struct llist_node pool_node; + struct { + /* + * reference counter for object management + */ + atomic_t ref; + + /* + * function causing this wait + */ + const char *wait_fn; + + /* + * the associated class + */ + struct dept_class *class; + + /* + * which IRQ the wait was placed in + */ + unsigned long irqf; + + /* + * where the IRQ wait happened + */ + unsigned long irq_ip[DEPT_IRQS_NR]; + struct dept_stack *irq_stack[DEPT_IRQS_NR]; + + /* + * where the wait happened + */ + unsigned long wait_ip; + struct dept_stack *wait_stack; + + /* + * whether this wait is for commit in scheduler + */ + bool sched_sleep; + }; + }; +}; + +struct dept_dep { + union { + struct llist_node pool_node; + struct { + /* + * reference counter for object management + */ + atomic_t ref; + + /* + * key data of dependency + */ + struct dept_ecxt *ecxt; + struct dept_wait *wait; + + /* + * This object can be referred without dept_lock + * held but with IRQ disabled, e.g. for hash + * lookup. So deferred deletion is needed. + */ + struct rcu_head rh; + + /* + * for hashing this object + */ + struct hlist_node hash_node; + + /* + * for linking to a class object + */ + struct list_head dep_node; + struct list_head dep_rev_node; + }; + }; +}; + +struct dept_hash { + /* + * hash table + */ + struct hlist_head *table; + + /* + * size of the table e.i. 2^bits + */ + int bits; +}; + +struct dept_ecxt_held { + /* + * associated event context + */ + struct dept_ecxt *ecxt; + + /* + * unique key for this dept_ecxt_held + */ + struct dept_map *map; + + /* + * class of the ecxt of this dept_ecxt_held + */ + struct dept_class *class; + + /* + * the wgen when the event context started + */ + unsigned int wgen; + + /* + * subclass that only works in the local context + */ + int sub_l; +}; + +struct dept_wait_hist { + /* + * associated wait + */ + struct dept_wait *wait; + + /* + * unique id of all waits system-wise until wrapped + */ + unsigned int wgen; + + /* + * local context id to identify IRQ context + */ + unsigned int ctxt_id; +}; + +extern void dept_on(void); +extern void dept_off(void); +extern void dept_init(void); +extern void dept_task_init(struct task_struct *t); +extern void dept_task_exit(struct task_struct *t); +extern void dept_free_range(void *start, unsigned int sz); + +extern void dept_map_init(struct dept_map *m, struct dept_key *k, int sub_= u, const char *n); +extern void dept_map_reinit(struct dept_map *m, struct dept_key *k, int su= b_u, const char *n); +extern void dept_map_copy(struct dept_map *to, struct dept_map *from); +extern void dept_wait(struct dept_map *m, unsigned long w_f, unsigned long= ip, const char *w_fn, int sub_l); +extern void dept_stage_wait(struct dept_map *m, struct dept_key *k, unsign= ed long ip, const char *w_fn); +extern void dept_request_event_wait_commit(void); +extern void dept_clean_stage(void); +extern void dept_ttwu_stage_wait(struct task_struct *t, unsigned long ip); +extern void dept_ecxt_enter(struct dept_map *m, unsigned long e_f, unsigne= d long ip, const char *c_fn, const char *e_fn, int sub_l); +extern bool dept_ecxt_holding(struct dept_map *m, unsigned long e_f); +extern void dept_request_event(struct dept_map *m); +extern void dept_event(struct dept_map *m, unsigned long e_f, unsigned lon= g ip, const char *e_fn); +extern void dept_ecxt_exit(struct dept_map *m, unsigned long e_f, unsigned= long ip); +extern void dept_sched_enter(void); +extern void dept_sched_exit(void); + +static inline void dept_ecxt_enter_nokeep(struct dept_map *m) +{ + dept_ecxt_enter(m, 0UL, 0UL, NULL, NULL, 0); +} + +/* + * for users who want to manage external keys + */ +extern void dept_key_init(struct dept_key *k); +extern void dept_key_destroy(struct dept_key *k); +extern void dept_map_ecxt_modify(struct dept_map *m, unsigned long e_f, st= ruct dept_key *new_k, unsigned long new_e_f, unsigned long new_ip, const ch= ar *new_c_fn, const char *new_e_fn, int new_sub_l); + +extern void dept_softirq_enter(void); +extern void dept_hardirq_enter(void); +extern void dept_softirqs_on_ip(unsigned long ip); +extern void dept_hardirqs_on(void); +extern void dept_softirqs_off(void); +extern void dept_hardirqs_off(void); +#else /* !CONFIG_DEPT */ +struct dept_key { }; +struct dept_map { }; + +#define DEPT_MAP_INITIALIZER(n, k) { } + +#define dept_on() do { } while (0) +#define dept_off() do { } while (0) +#define dept_init() do { } while (0) +#define dept_task_init(t) do { } while (0) +#define dept_task_exit(t) do { } while (0) +#define dept_free_range(s, sz) do { } while (0) + +#define dept_map_init(m, k, su, n) do { (void)(n); (void)(k); } while (0) +#define dept_map_reinit(m, k, su, n) do { (void)(n); (void)(k); } while = (0) +#define dept_map_copy(t, f) do { } while (0) +#define dept_wait(m, w_f, ip, w_fn, sl) do { (void)(w_fn); } while (0) +#define dept_stage_wait(m, k, ip, w_fn) do { (void)(k); (void)(w_fn); } = while (0) +#define dept_request_event_wait_commit() do { } while (0) +#define dept_clean_stage() do { } while (0) +#define dept_ttwu_stage_wait(t, ip) do { } while (0) +#define dept_ecxt_enter(m, e_f, ip, c_fn, e_fn, sl) do { (void)(c_fn); (vo= id)(e_fn); } while (0) +#define dept_ecxt_holding(m, e_f) false +#define dept_request_event(m) do { } while (0) +#define dept_event(m, e_f, ip, e_fn) do { (void)(e_fn); } while (0) +#define dept_ecxt_exit(m, e_f, ip) do { } while (0) +#define dept_sched_enter() do { } while (0) +#define dept_sched_exit() do { } while (0) +#define dept_ecxt_enter_nokeep(m) do { } while (0) +#define dept_key_init(k) do { (void)(k); } while (0) +#define dept_key_destroy(k) do { (void)(k); } while (0) +#define dept_map_ecxt_modify(m, e_f, n_k, n_e_f, n_ip, n_c_fn, n_e_fn, n_s= l) do { (void)(n_k); (void)(n_c_fn); (void)(n_e_fn); } while (0) + +#define dept_softirq_enter() do { } while (0) +#define dept_hardirq_enter() do { } while (0) +#define dept_softirqs_on_ip(ip) do { } while (0) +#define dept_hardirqs_on() do { } while (0) +#define dept_softirqs_off() do { } while (0) +#define dept_hardirqs_off() do { } while (0) +#endif +#endif /* __LINUX_DEPT_H */ diff --git a/include/linux/hardirq.h b/include/linux/hardirq.h index d57cab4d4c06..bb279dbbe748 100644 --- a/include/linux/hardirq.h +++ b/include/linux/hardirq.h @@ -5,6 +5,7 @@ #include #include #include +#include #include #include #include @@ -106,6 +107,7 @@ void irq_exit_rcu(void); */ #define __nmi_enter() \ do { \ + dept_off(); \ lockdep_off(); \ arch_nmi_enter(); \ BUG_ON(in_nmi() =3D=3D NMI_MASK); \ @@ -128,6 +130,7 @@ void irq_exit_rcu(void); __preempt_count_sub(NMI_OFFSET + HARDIRQ_OFFSET); \ arch_nmi_exit(); \ lockdep_on(); \ + dept_on(); \ } while (0) =20 #define nmi_exit() \ diff --git a/include/linux/sched.h b/include/linux/sched.h index f96ac1982893..a1924b40feb5 100644 --- a/include/linux/sched.h +++ b/include/linux/sched.h @@ -48,6 +48,8 @@ #include #include #include +#include +#include =20 /* task_struct member predeclarations (sorted alphabetically): */ struct audit_context; @@ -810,6 +812,110 @@ struct kmap_ctrl { #endif }; =20 +#ifdef CONFIG_DEPT +struct dept_task { + /* + * all event contexts that have entered and before exiting + */ + struct dept_ecxt_held ecxt_held[DEPT_MAX_ECXT_HELD]; + int ecxt_held_pos; + + /* + * ring buffer holding all waits that have happened + */ + struct dept_wait_hist wait_hist[DEPT_MAX_WAIT_HIST]; + int wait_hist_pos; + + /* + * sequential id to identify each IRQ context + */ + unsigned int irq_id[DEPT_IRQS_NR]; + + /* + * for tracking IRQ-enabled points with cross-event + */ + unsigned int wgen_enirq[DEPT_IRQS_NR]; + + /* + * for keeping up-to-date IRQ-enabled points + */ + unsigned long enirq_ip[DEPT_IRQS_NR]; + + /* + * for reserving a current stack instance at each operation + */ + struct dept_stack *stack; + + /* + * for preventing recursive call into DEPT engine + */ + int recursive; + + /* + * for preventing reentrance to WARN*() while warning + */ + int in_warning; + + /* + * for staging data to commit a wait + */ + struct dept_map stage_m; + struct dept_map *stage_real_m; + bool stage_sched_map; + const char *stage_w_fn; + unsigned long stage_ip; + arch_spinlock_t stage_lock; + + /* + * the number of missing ecxts + */ + int missing_ecxt; + + /* + * for tracking IRQ-enable state + */ + bool hardirqs_enabled; + bool softirqs_enabled; + + /* + * whether the current is on do_exit() + */ + bool task_exit; + + /* + * whether the current is running __schedule() + */ + bool in_sched; +}; + +#define DEPT_TASK_INITIALIZER(t) \ +{ \ + .wait_hist =3D { { .wait =3D NULL, } }, \ + .ecxt_held_pos =3D 0, \ + .wait_hist_pos =3D 0, \ + .irq_id =3D { 0U }, \ + .wgen_enirq =3D { 0U }, \ + .enirq_ip =3D { 0UL }, \ + .stack =3D NULL, \ + .recursive =3D 0, \ + .in_warning =3D 0, \ + .stage_m =3D DEPT_MAP_INITIALIZER((t)->stage_m, NULL), \ + .stage_real_m =3D NULL, \ + .stage_sched_map =3D false, \ + .stage_w_fn =3D NULL, \ + .stage_ip =3D 0UL, \ + .stage_lock =3D (arch_spinlock_t)__ARCH_SPIN_LOCK_UNLOCKED,\ + .missing_ecxt =3D 0, \ + .hardirqs_enabled =3D false, \ + .softirqs_enabled =3D false, \ + .task_exit =3D false, \ + .in_sched =3D false, \ +} +#else +struct dept_task { }; +#define DEPT_TASK_INITIALIZER(t) { } +#endif + struct task_struct { #ifdef CONFIG_THREAD_INFO_IN_TASK /* @@ -1267,6 +1373,8 @@ struct task_struct { struct held_lock held_locks[MAX_LOCK_DEPTH]; #endif =20 + struct dept_task dept_task; + #if defined(CONFIG_UBSAN) && !defined(CONFIG_UBSAN_TRAP) unsigned int in_ubsan; #endif @@ -2210,8 +2318,6 @@ extern bool sched_task_on_rq(struct task_struct *p); extern unsigned long get_wchan(struct task_struct *p); extern struct task_struct *cpu_curr_snapshot(int cpu); =20 -#include - /* * In order to reduce various lock holder preemption latencies provide an * interface to see if a vCPU is currently running or not. diff --git a/init/init_task.c b/init/init_task.c index e557f622bd90..84da2464c390 100644 --- a/init/init_task.c +++ b/init/init_task.c @@ -14,6 +14,7 @@ #include #include #include +#include =20 #include =20 @@ -204,6 +205,7 @@ struct task_struct init_task __aligned(L1_CACHE_BYTES) = =3D { .curr_chain_key =3D INITIAL_CHAIN_KEY, .lockdep_recursion =3D 0, #endif + .dept_task =3D DEPT_TASK_INITIALIZER(init_task), #ifdef CONFIG_FUNCTION_GRAPH_TRACER .ret_stack =3D NULL, .tracing_graph_pause =3D ATOMIC_INIT(0), diff --git a/init/main.c b/init/main.c index 7f0a2a3dbd29..9b91f4d3b518 100644 --- a/init/main.c +++ b/init/main.c @@ -65,6 +65,7 @@ #include #include #include +#include #include #include #include @@ -1037,6 +1038,7 @@ void start_kernel(void) panic_param); =20 lockdep_init(); + dept_init(); =20 /* * Need to run this when irqs are enabled, because it wants diff --git a/kernel/Makefile b/kernel/Makefile index 434929de17ef..ee83774d4c73 100644 --- a/kernel/Makefile +++ b/kernel/Makefile @@ -56,6 +56,7 @@ obj-y +=3D livepatch/ obj-y +=3D dma/ obj-y +=3D entry/ obj-$(CONFIG_MODULES) +=3D module/ +obj-y +=3D dependency/ =20 obj-$(CONFIG_KCMP) +=3D kcmp.o obj-$(CONFIG_FREEZER) +=3D freezer.o diff --git a/kernel/dependency/Makefile b/kernel/dependency/Makefile new file mode 100644 index 000000000000..b5cfb8a03c0c --- /dev/null +++ b/kernel/dependency/Makefile @@ -0,0 +1,3 @@ +# SPDX-License-Identifier: GPL-2.0 + +obj-$(CONFIG_DEPT) +=3D dept.o diff --git a/kernel/dependency/dept.c b/kernel/dependency/dept.c new file mode 100644 index 000000000000..a6671b13891f --- /dev/null +++ b/kernel/dependency/dept.c @@ -0,0 +1,3001 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * DEPT(DEPendency Tracker) - Runtime dependency tracker + * + * Started by Byungchul Park : + * + * Copyright (c) 2020 LG Electronics, Inc., Byungchul Park + * Copyright (c) 2024 SK hynix, Inc., Byungchul Park + * + * DEPT provides a general way to detect potential deadlocks at runtime + * and the interest is not limited to typical lock but to every + * synchronization primitives. + * + * The following ideas were borrowed from LOCKDEP: + * + * 1) Use a graph to track relationship between classes. + * 2) Prevent performance regression using hash. + * + * The following items were enhanced from LOCKDEP: + * + * 1) Cover more deadlock cases. + * 2) Allow multiple reports. + * + * TODO: Both LOCKDEP and DEPT should co-exist until DEPT is considered + * stable. Then the dependency check routine should be replaced with + * DEPT after. It should finally look like: + * + * + * + * As is: + * + * LOCKDEP + * +-----------------------------------------+ + * | Lock usage correctness check | <-> locks + * | | + * | | + * | +-------------------------------------+ | + * | | Dependency check | | + * | | (by tracking lock acquisition order)| | + * | +-------------------------------------+ | + * | | + * +-----------------------------------------+ + * + * DEPT + * +-----------------------------------------+ + * | Dependency check | <-> waits/events + * | (by tracking wait and event context) | + * +-----------------------------------------+ + * + * + * + * To be: + * + * LOCKDEP + * +-----------------------------------------+ + * | Lock usage correctness check | <-> locks + * | | + * | | + * | (Request dependency check) | + * | T | + * +--------------------|--------------------+ + * | + * DEPT V + * +-----------------------------------------+ + * | Dependency check | <-> waits/events + * | (by tracking wait and event context) | + * +-----------------------------------------+ + */ + +#include +#include +#include +#include +#include +#include +#include +#include + +static int dept_stop; +static int dept_per_cpu_ready; + +static inline struct dept_task *dept_task(void) +{ + return ¤t->dept_task; +} + +#define DEPT_READY_WARN (!oops_in_progress && !dept_task()->in_warning) + +/* + * Make all operations using DEPT_WARN_ON() fail on oops_in_progress and + * prevent warning message. + */ +#define DEPT_WARN_ON_ONCE(c) \ + ({ \ + int __ret =3D !!(c); \ + \ + if (likely(DEPT_READY_WARN)) { \ + ++dept_task()->in_warning; \ + WARN_ONCE(c, "DEPT_WARN_ON_ONCE: " #c); \ + --dept_task()->in_warning; \ + } \ + __ret; \ + }) + +#define DEPT_WARN_ONCE(s...) \ + ({ \ + if (likely(DEPT_READY_WARN)) { \ + ++dept_task()->in_warning; \ + WARN_ONCE(1, "DEPT_WARN_ONCE: " s); \ + --dept_task()->in_warning; \ + } \ + }) + +#define DEPT_WARN_ON(c) \ + ({ \ + int __ret =3D !!(c); \ + \ + if (likely(DEPT_READY_WARN)) { \ + ++dept_task()->in_warning; \ + WARN(c, "DEPT_WARN_ON: " #c); \ + --dept_task()->in_warning; \ + } \ + __ret; \ + }) + +#define DEPT_WARN(s...) \ + ({ \ + if (likely(DEPT_READY_WARN)) { \ + ++dept_task()->in_warning; \ + WARN(1, "DEPT_WARN: " s); \ + --dept_task()->in_warning; \ + } \ + }) + +#define DEPT_STOP(s...) \ + ({ \ + WRITE_ONCE(dept_stop, 1); \ + if (likely(DEPT_READY_WARN)) { \ + ++dept_task()->in_warning; \ + WARN(1, "DEPT_STOP: " s); \ + --dept_task()->in_warning; \ + } \ + }) + +#define DEPT_INFO_ONCE(s...) pr_warn_once("DEPT_INFO_ONCE: " s) + +static arch_spinlock_t dept_spin =3D (arch_spinlock_t)__ARCH_SPIN_LOCK_UNL= OCKED; + +/* + * DEPT internal engine should be cautious in using outside functions + * e.g. printk at reporting since that kind of usage might cause + * untrackable deadlock. + */ +static atomic_t dept_outworld =3D ATOMIC_INIT(0); + +static void dept_outworld_enter(void) +{ + atomic_inc(&dept_outworld); +} + +static void dept_outworld_exit(void) +{ + atomic_dec(&dept_outworld); +} + +static bool dept_outworld_entered(void) +{ + return atomic_read(&dept_outworld); +} + +static bool dept_lock(void) +{ + while (!arch_spin_trylock(&dept_spin)) + if (unlikely(dept_outworld_entered())) + return false; + return true; +} + +static void dept_unlock(void) +{ + arch_spin_unlock(&dept_spin); +} + +enum bfs_ret { + BFS_CONTINUE, + BFS_DONE, + BFS_SKIP, +}; + +static bool before(unsigned int a, unsigned int b) +{ + return (int)(a - b) < 0; +} + +static bool valid_stack(struct dept_stack *s) +{ + return s && s->nr > 0; +} + +static bool valid_class(struct dept_class *c) +{ + return c->key; +} + +static void invalidate_class(struct dept_class *c) +{ + c->key =3D 0UL; +} + +static struct dept_ecxt *dep_e(struct dept_dep *d) +{ + return d->ecxt; +} + +static struct dept_wait *dep_w(struct dept_dep *d) +{ + return d->wait; +} + +static struct dept_class *dep_fc(struct dept_dep *d) +{ + return dep_e(d)->class; +} + +static struct dept_class *dep_tc(struct dept_dep *d) +{ + return dep_w(d)->class; +} + +static const char *irq_str(int irq) +{ + if (irq =3D=3D DEPT_SIRQ) + return "softirq"; + if (irq =3D=3D DEPT_HIRQ) + return "hardirq"; + return "(unknown)"; +} + +/* + * Dept doesn't work either when it's stopped by DEPT_STOP() or in a nmi + * context. + */ +static bool dept_working(void) +{ + return !READ_ONCE(dept_stop) && !in_nmi(); +} + +/* + * Even k =3D=3D NULL is considered as a valid key because it would use + * &->map_key as the key in that case. + */ +struct dept_key __dept_no_validate__; +static bool valid_key(struct dept_key *k) +{ + return &__dept_no_validate__ !=3D k; +} + +/* + * Pool + * =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D + * DEPT maintains pools to provide objects in a safe way. + * + * 1) Static pool is used at the beginning of booting time. + * 2) Local pool is tried first before the static pool. Objects that + * have been freed will be placed. + */ + +enum object_t { +#define OBJECT(id, nr) OBJECT_##id, + #include "dept_object.h" +#undef OBJECT + OBJECT_NR, +}; + +#define OBJECT(id, nr) \ +static struct dept_##id spool_##id[nr]; \ +static DEFINE_PER_CPU(struct llist_head, lpool_##id); + #include "dept_object.h" +#undef OBJECT + +struct dept_pool { + const char *name; + + /* + * object size + */ + size_t obj_sz; + + /* + * the number of the static array + */ + atomic_t obj_nr; + + /* + * offset of ->pool_node + */ + size_t node_off; + + /* + * pointer to the pool + */ + void *spool; + struct llist_head boot_pool; + struct llist_head __percpu *lpool; +}; + +static struct dept_pool pool[OBJECT_NR] =3D { +#define OBJECT(id, nr) { \ + .name =3D #id, \ + .obj_sz =3D sizeof(struct dept_##id), \ + .obj_nr =3D ATOMIC_INIT(nr), \ + .node_off =3D offsetof(struct dept_##id, pool_node), \ + .spool =3D spool_##id, \ + .lpool =3D &lpool_##id, }, + #include "dept_object.h" +#undef OBJECT +}; + +/* + * Can use llist no matter whether CONFIG_ARCH_HAVE_NMI_SAFE_CMPXCHG is + * enabled or not because NMI and other contexts in the same CPU never + * run inside of DEPT concurrently by preventing reentrance. + */ +static void *from_pool(enum object_t t) +{ + struct dept_pool *p; + struct llist_head *h; + struct llist_node *n; + + /* + * llist_del_first() doesn't allow concurrent access e.g. + * between process and IRQ context. + */ + if (DEPT_WARN_ON(!irqs_disabled())) + return NULL; + + p =3D &pool[t]; + + /* + * Try local pool first. + */ + if (likely(dept_per_cpu_ready)) + h =3D this_cpu_ptr(p->lpool); + else + h =3D &p->boot_pool; + + n =3D llist_del_first(h); + if (n) + return (void *)n - p->node_off; + + /* + * Try static pool. + */ + if (atomic_read(&p->obj_nr) > 0) { + int idx =3D atomic_dec_return(&p->obj_nr); + + if (idx >=3D 0) + return p->spool + (idx * p->obj_sz); + } + + DEPT_INFO_ONCE("---------------------------------------------\n" + " Some of Dept internal resources are run out.\n" + " Dept might still work if the resources get freed.\n" + " However, the chances are Dept will suffer from\n" + " the lack from now. Needs to extend the internal\n" + " resource pools. Ask max.byungchul.park@gmail.com\n"); + return NULL; +} + +static void to_pool(void *o, enum object_t t) +{ + struct dept_pool *p =3D &pool[t]; + struct llist_head *h; + + preempt_disable(); + if (likely(dept_per_cpu_ready)) + h =3D this_cpu_ptr(p->lpool); + else + h =3D &p->boot_pool; + + llist_add(o + p->node_off, h); + preempt_enable(); +} + +#define OBJECT(id, nr) \ +static void (*ctor_##id)(struct dept_##id *a); \ +static void (*dtor_##id)(struct dept_##id *a); \ +static struct dept_##id *new_##id(void) \ +{ \ + struct dept_##id *a; \ + \ + a =3D (struct dept_##id *)from_pool(OBJECT_##id); \ + if (unlikely(!a)) \ + return NULL; \ + \ + atomic_set(&a->ref, 1); \ + \ + if (ctor_##id) \ + ctor_##id(a); \ + \ + return a; \ +} \ + \ +static struct dept_##id *get_##id(struct dept_##id *a) \ +{ \ + atomic_inc(&a->ref); \ + return a; \ +} \ + \ +static void put_##id(struct dept_##id *a) \ +{ \ + if (!atomic_dec_return(&a->ref)) { \ + if (dtor_##id) \ + dtor_##id(a); \ + to_pool(a, OBJECT_##id); \ + } \ +} \ + \ +static void del_##id(struct dept_##id *a) \ +{ \ + put_##id(a); \ +} \ + \ +static bool __maybe_unused id##_consumed(struct dept_##id *a) \ +{ \ + return a && atomic_read(&a->ref) > 1; \ +} +#include "dept_object.h" +#undef OBJECT + +#define SET_CONSTRUCTOR(id, f) \ +static void (*ctor_##id)(struct dept_##id *a) =3D f + +static void initialize_dep(struct dept_dep *d) +{ + INIT_LIST_HEAD(&d->dep_node); + INIT_LIST_HEAD(&d->dep_rev_node); +} +SET_CONSTRUCTOR(dep, initialize_dep); + +static void initialize_class(struct dept_class *c) +{ + int i; + + for (i =3D 0; i < DEPT_IRQS_NR; i++) { + struct dept_iecxt *ie =3D &c->iecxt[i]; + struct dept_iwait *iw =3D &c->iwait[i]; + + ie->ecxt =3D NULL; + ie->enirq =3D i; + ie->staled =3D false; + + iw->wait =3D NULL; + iw->irq =3D i; + iw->staled =3D false; + iw->touched =3D false; + } + c->bfs_gen =3D 0U; + + INIT_LIST_HEAD(&c->all_node); + INIT_LIST_HEAD(&c->dep_head); + INIT_LIST_HEAD(&c->dep_rev_head); + INIT_LIST_HEAD(&c->bfs_node); +} +SET_CONSTRUCTOR(class, initialize_class); + +static void initialize_ecxt(struct dept_ecxt *e) +{ + int i; + + for (i =3D 0; i < DEPT_IRQS_NR; i++) { + e->enirq_stack[i] =3D NULL; + e->enirq_ip[i] =3D 0UL; + } + e->ecxt_ip =3D 0UL; + e->ecxt_stack =3D NULL; + e->enirqf =3D 0UL; + e->event_ip =3D 0UL; + e->event_stack =3D NULL; +} +SET_CONSTRUCTOR(ecxt, initialize_ecxt); + +static void initialize_wait(struct dept_wait *w) +{ + int i; + + for (i =3D 0; i < DEPT_IRQS_NR; i++) { + w->irq_stack[i] =3D NULL; + w->irq_ip[i] =3D 0UL; + } + w->wait_ip =3D 0UL; + w->wait_stack =3D NULL; + w->irqf =3D 0UL; +} +SET_CONSTRUCTOR(wait, initialize_wait); + +static void initialize_stack(struct dept_stack *s) +{ + s->nr =3D 0; +} +SET_CONSTRUCTOR(stack, initialize_stack); + +#define OBJECT(id, nr) \ +static void (*ctor_##id)(struct dept_##id *a); + #include "dept_object.h" +#undef OBJECT + +#undef SET_CONSTRUCTOR + +#define SET_DESTRUCTOR(id, f) \ +static void (*dtor_##id)(struct dept_##id *a) =3D f + +static void destroy_dep(struct dept_dep *d) +{ + if (dep_e(d)) + put_ecxt(dep_e(d)); + if (dep_w(d)) + put_wait(dep_w(d)); +} +SET_DESTRUCTOR(dep, destroy_dep); + +static void destroy_ecxt(struct dept_ecxt *e) +{ + int i; + + for (i =3D 0; i < DEPT_IRQS_NR; i++) + if (e->enirq_stack[i]) + put_stack(e->enirq_stack[i]); + if (e->class) + put_class(e->class); + if (e->ecxt_stack) + put_stack(e->ecxt_stack); + if (e->event_stack) + put_stack(e->event_stack); +} +SET_DESTRUCTOR(ecxt, destroy_ecxt); + +static void destroy_wait(struct dept_wait *w) +{ + int i; + + for (i =3D 0; i < DEPT_IRQS_NR; i++) + if (w->irq_stack[i]) + put_stack(w->irq_stack[i]); + if (w->class) + put_class(w->class); + if (w->wait_stack) + put_stack(w->wait_stack); +} +SET_DESTRUCTOR(wait, destroy_wait); + +#define OBJECT(id, nr) \ +static void (*dtor_##id)(struct dept_##id *a); + #include "dept_object.h" +#undef OBJECT + +#undef SET_DESTRUCTOR + +/* + * Caching and hashing + * =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D + * DEPT makes use of caching and hashing to improve performance. Each + * object can be obtained in O(1) with its key. + * + * NOTE: Currently we assume all the objects in the hashs will never be + * removed. Implement it when needed. + */ + +/* + * Some information might be lost but it's only for hashing key. + */ +static unsigned long mix(unsigned long a, unsigned long b) +{ + int halfbits =3D sizeof(unsigned long) * 8 / 2; + unsigned long halfmask =3D (1UL << halfbits) - 1UL; + + return (a << halfbits) | (b & halfmask); +} + +static bool cmp_dep(struct dept_dep *d1, struct dept_dep *d2) +{ + return dep_fc(d1)->key =3D=3D dep_fc(d2)->key && + dep_tc(d1)->key =3D=3D dep_tc(d2)->key; +} + +static unsigned long key_dep(struct dept_dep *d) +{ + return mix(dep_fc(d)->key, dep_tc(d)->key); +} + +static bool cmp_class(struct dept_class *c1, struct dept_class *c2) +{ + return c1->key =3D=3D c2->key; +} + +static unsigned long key_class(struct dept_class *c) +{ + return c->key; +} + +#define HASH(id, bits) \ +static struct hlist_head table_##id[1 << (bits)]; \ + \ +static struct hlist_head *head_##id(struct dept_##id *a) \ +{ \ + return table_##id + hash_long(key_##id(a), bits); \ +} \ + \ +static struct dept_##id *hash_lookup_##id(struct dept_##id *a) \ +{ \ + struct dept_##id *b; \ + \ + hlist_for_each_entry_rcu(b, head_##id(a), hash_node) \ + if (cmp_##id(a, b)) \ + return b; \ + return NULL; \ +} \ + \ +static void hash_add_##id(struct dept_##id *a) \ +{ \ + get_##id(a); \ + hlist_add_head_rcu(&a->hash_node, head_##id(a)); \ +} \ + \ +static void hash_del_##id(struct dept_##id *a) \ +{ \ + hlist_del_rcu(&a->hash_node); \ + put_##id(a); \ +} +#include "dept_hash.h" +#undef HASH + +static struct dept_dep *lookup_dep(struct dept_class *fc, + struct dept_class *tc) +{ + struct dept_ecxt onetime_e =3D { .class =3D fc }; + struct dept_wait onetime_w =3D { .class =3D tc }; + struct dept_dep onetime_d =3D { .ecxt =3D &onetime_e, + .wait =3D &onetime_w }; + return hash_lookup_dep(&onetime_d); +} + +static struct dept_class *lookup_class(unsigned long key) +{ + struct dept_class onetime_c =3D { .key =3D key }; + + return hash_lookup_class(&onetime_c); +} + +/* + * Report + * =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D + * DEPT prints useful information to help debugging on detection of + * problematic dependency. + */ + +static void print_ip_stack(unsigned long ip, struct dept_stack *s) +{ + if (ip) + print_ip_sym(KERN_WARNING, ip); + +#ifdef CONFIG_DEPT_DEBUG + if (!s) + pr_warn("stack is NULL.\n"); + else if (!s->nr) + pr_warn("stack->nr is 0.\n"); + if (s) + pr_warn("stack ref is %d.\n", atomic_read(&s->ref)); +#endif + + if (valid_stack(s)) { + pr_warn("stacktrace:\n"); + stack_trace_print(s->raw, s->nr, 5); + } + + if (!ip && !valid_stack(s)) + pr_warn("(N/A)\n"); +} + +#define print_spc(spc, fmt, ...) \ + pr_warn("%*c" fmt, (spc) * 3, ' ', ##__VA_ARGS__) + +static void print_diagram(struct dept_dep *d) +{ + struct dept_ecxt *e =3D dep_e(d); + struct dept_wait *w =3D dep_w(d); + struct dept_class *fc =3D dep_fc(d); + struct dept_class *tc =3D dep_tc(d); + unsigned long irqf; + int irq; + bool firstline =3D true; + int spc =3D 1; + const char *w_fn =3D w->wait_fn ?: "(unknown)"; + const char *e_fn =3D e->event_fn ?: "(unknown)"; + const char *c_fn =3D e->ecxt_fn ?: "(unknown)"; + const char *fc_n =3D fc->sched_map ? "" : (fc->name ?: "(unknown)"= ); + const char *tc_n =3D tc->sched_map ? "" : (tc->name ?: "(unknown)"= ); + + irqf =3D e->enirqf & w->irqf; + for_each_set_bit(irq, &irqf, DEPT_IRQS_NR) { + if (!firstline) + pr_warn("\nor\n\n"); + firstline =3D false; + + print_spc(spc, "[S] %s(%s:%d)\n", c_fn, fc_n, fc->sub_id); + print_spc(spc, " <%s interrupt>\n", irq_str(irq)); + print_spc(spc + 1, "[W] %s(%s:%d)\n", w_fn, tc_n, tc->sub_id); + print_spc(spc, "[E] %s(%s:%d)\n", e_fn, fc_n, fc->sub_id); + } + + if (!irqf) { + print_spc(spc, "[S] %s(%s:%d)\n", c_fn, fc_n, fc->sub_id); + print_spc(spc, "[W] %s(%s:%d)\n", w_fn, tc_n, tc->sub_id); + print_spc(spc, "[E] %s(%s:%d)\n", e_fn, fc_n, fc->sub_id); + } +} + +static void print_dep(struct dept_dep *d) +{ + struct dept_ecxt *e =3D dep_e(d); + struct dept_wait *w =3D dep_w(d); + struct dept_class *fc =3D dep_fc(d); + struct dept_class *tc =3D dep_tc(d); + unsigned long irqf; + int irq; + const char *w_fn =3D w->wait_fn ?: "(unknown)"; + const char *e_fn =3D e->event_fn ?: "(unknown)"; + const char *c_fn =3D e->ecxt_fn ?: "(unknown)"; + const char *fc_n =3D fc->sched_map ? "" : (fc->name ?: "(unknown)"= ); + const char *tc_n =3D tc->sched_map ? "" : (tc->name ?: "(unknown)"= ); + + irqf =3D e->enirqf & w->irqf; + for_each_set_bit(irq, &irqf, DEPT_IRQS_NR) { + pr_warn("%s has been enabled:\n", irq_str(irq)); + print_ip_stack(e->enirq_ip[irq], e->enirq_stack[irq]); + pr_warn("\n"); + + pr_warn("[S] %s(%s:%d):\n", c_fn, fc_n, fc->sub_id); + print_ip_stack(e->ecxt_ip, e->ecxt_stack); + pr_warn("\n"); + + pr_warn("[W] %s(%s:%d) in %s context:\n", + w_fn, tc_n, tc->sub_id, irq_str(irq)); + print_ip_stack(w->irq_ip[irq], w->irq_stack[irq]); + pr_warn("\n"); + + pr_warn("[E] %s(%s:%d):\n", e_fn, fc_n, fc->sub_id); + print_ip_stack(e->event_ip, e->event_stack); + } + + if (!irqf) { + pr_warn("[S] %s(%s:%d):\n", c_fn, fc_n, fc->sub_id); + print_ip_stack(e->ecxt_ip, e->ecxt_stack); + pr_warn("\n"); + + pr_warn("[W] %s(%s:%d):\n", w_fn, tc_n, tc->sub_id); + print_ip_stack(w->wait_ip, w->wait_stack); + pr_warn("\n"); + + pr_warn("[E] %s(%s:%d):\n", e_fn, fc_n, fc->sub_id); + print_ip_stack(e->event_ip, e->event_stack); + } +} + +static void save_current_stack(int skip); + +/* + * Print all classes in a circle. + */ +static void print_circle(struct dept_class *c) +{ + struct dept_class *fc =3D c->bfs_parent; + struct dept_class *tc =3D c; + int i; + + dept_outworld_enter(); + save_current_stack(6); + + pr_warn("=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D\n"); + pr_warn("DEPT: Circular dependency has been detected.\n"); + pr_warn("%s %.*s %s\n", init_utsname()->release, + (int)strcspn(init_utsname()->version, " "), + init_utsname()->version, + print_tainted()); + pr_warn("---------------------------------------------------\n"); + pr_warn("summary\n"); + pr_warn("---------------------------------------------------\n"); + + if (fc =3D=3D tc) + pr_warn("*** AA DEADLOCK ***\n\n"); + else + pr_warn("*** DEADLOCK ***\n\n"); + + i =3D 0; + do { + struct dept_dep *d =3D lookup_dep(fc, tc); + + pr_warn("context %c\n", 'A' + (i++)); + print_diagram(d); + if (fc !=3D c) + pr_warn("\n"); + + tc =3D fc; + fc =3D fc->bfs_parent; + } while (tc !=3D c); + + pr_warn("\n"); + pr_warn("[S]: start of the event context\n"); + pr_warn("[W]: the wait blocked\n"); + pr_warn("[E]: the event not reachable\n"); + + i =3D 0; + do { + struct dept_dep *d =3D lookup_dep(fc, tc); + + pr_warn("---------------------------------------------------\n"); + pr_warn("context %c's detail\n", 'A' + i); + pr_warn("---------------------------------------------------\n"); + pr_warn("context %c\n", 'A' + (i++)); + print_diagram(d); + pr_warn("\n"); + print_dep(d); + + tc =3D fc; + fc =3D fc->bfs_parent; + } while (tc !=3D c); + + pr_warn("---------------------------------------------------\n"); + pr_warn("information that might be helpful\n"); + pr_warn("---------------------------------------------------\n"); + dump_stack(); + + dept_outworld_exit(); +} + +/* + * BFS(Breadth First Search) + * =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D + * Whenever a new dependency is added into the graph, search the graph + * for a new circular dependency. + */ + +struct bfs_ops { + void (*bfs_init)(void *, void *, void **); + void (*extend)(struct list_head *, void *); + void *(*dequeue)(struct list_head *); + enum bfs_ret (*callback)(void *, void *, void **); +}; + +static unsigned int bfs_gen; + +/* + * NOTE: Must be called with dept_lock held. + */ +static void bfs(void *root, struct bfs_ops *ops, void *in, void **out) +{ + LIST_HEAD(q); + enum bfs_ret ret; + + if (DEPT_WARN_ON(!ops || !ops->bfs_init || !ops->extend || + !ops->dequeue || !ops->callback)) + return; + + /* + * Avoid zero bfs_gen. + */ + bfs_gen =3D bfs_gen + 1 ?: 1; + ops->bfs_init(root, in, out); + + ret =3D ops->callback(root, in, out); + if (ret !=3D BFS_CONTINUE) + return; + + ops->extend(&q, root); + while (!list_empty(&q)) { + void *node =3D ops->dequeue(&q); + + if (ret =3D=3D BFS_DONE) + continue; + + ret =3D ops->callback(node, in, out); + if (ret =3D=3D BFS_CONTINUE) + ops->extend(&q, node); + } +} + +/* + * Main operations + * =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D + * Add dependencies - Each new dependency is added into the graph and + * checked if it forms a circular dependency. + * + * Track waits - Waits are queued into the ring buffer for later use to + * generate appropriate dependencies with cross-event. + * + * Track event contexts(ecxt) - Event contexts are pushed into local + * stack for later use to generate appropriate dependencies with waits. + */ + +static unsigned long cur_enirqf(void); +static int cur_irq(void); +static unsigned int cur_ctxt_id(void); + +static struct dept_iecxt *iecxt(struct dept_class *c, int irq) +{ + return &c->iecxt[irq]; +} + +static struct dept_iwait *iwait(struct dept_class *c, int irq) +{ + return &c->iwait[irq]; +} + +static void stale_iecxt(struct dept_iecxt *ie) +{ + if (ie->ecxt) + put_ecxt(ie->ecxt); + + WRITE_ONCE(ie->ecxt, NULL); + WRITE_ONCE(ie->staled, true); +} + +static void set_iecxt(struct dept_iecxt *ie, struct dept_ecxt *e) +{ + /* + * ->ecxt will never be updated once getting set until the class + * gets removed. + */ + if (ie->ecxt) + DEPT_WARN_ON(1); + else + WRITE_ONCE(ie->ecxt, get_ecxt(e)); +} + +static void stale_iwait(struct dept_iwait *iw) +{ + if (iw->wait) + put_wait(iw->wait); + + WRITE_ONCE(iw->wait, NULL); + WRITE_ONCE(iw->staled, true); +} + +static void set_iwait(struct dept_iwait *iw, struct dept_wait *w) +{ + /* + * ->wait will never be updated once getting set until the class + * gets removed. + */ + if (iw->wait) + DEPT_WARN_ON(1); + else + WRITE_ONCE(iw->wait, get_wait(w)); + + iw->touched =3D true; +} + +static void touch_iwait(struct dept_iwait *iw) +{ + iw->touched =3D true; +} + +static void untouch_iwait(struct dept_iwait *iw) +{ + iw->touched =3D false; +} + +static struct dept_stack *get_current_stack(void) +{ + struct dept_stack *s =3D dept_task()->stack; + + return s ? get_stack(s) : NULL; +} + +static void prepare_current_stack(void) +{ + DEPT_WARN_ON(dept_task()->stack); + + dept_task()->stack =3D new_stack(); +} + +static void save_current_stack(int skip) +{ + struct dept_stack *s =3D dept_task()->stack; + + if (!s) + return; + + if (valid_stack(s)) + return; + + s->nr =3D stack_trace_save(s->raw, DEPT_MAX_STACK_ENTRY, skip); +} + +static void finish_current_stack(void) +{ + struct dept_stack *s =3D dept_task()->stack; + + /* + * Fill the struct dept_stack with a valid stracktrace if it has + * been referred at least once. + */ + if (stack_consumed(s)) + save_current_stack(2); + + dept_task()->stack =3D NULL; + + /* + * Actual deletion will happen at put_stack() if the stack has + * been referred. + */ + if (s) + del_stack(s); +} + +/* + * FIXME: For now, disable LOCKDEP while DEPT is working. + * + * Both LOCKDEP and DEPT report it on a deadlock detection using + * printk taking the risk of another deadlock that might be caused by + * locks of console or printk between inside and outside of them. + * + * For DEPT, it's no problem since multiple reports are allowed. But it + * would be a bad idea for LOCKDEP since it will stop even on a singe + * report. So we need to prevent LOCKDEP from its reporting the risk + * DEPT would take when reporting something. + */ +#include + +void noinstr dept_off(void) +{ + dept_task()->recursive++; + lockdep_off(); +} + +void noinstr dept_on(void) +{ + lockdep_on(); + dept_task()->recursive--; +} + +static unsigned long dept_enter(void) +{ + unsigned long flags; + + flags =3D arch_local_irq_save(); + dept_off(); + prepare_current_stack(); + return flags; +} + +static void dept_exit(unsigned long flags) +{ + finish_current_stack(); + dept_on(); + arch_local_irq_restore(flags); +} + +static unsigned long dept_enter_recursive(void) +{ + unsigned long flags; + + flags =3D arch_local_irq_save(); + return flags; +} + +static void dept_exit_recursive(unsigned long flags) +{ + arch_local_irq_restore(flags); +} + +/* + * NOTE: Must be called with dept_lock held. + */ +static struct dept_dep *__add_dep(struct dept_ecxt *e, + struct dept_wait *w) +{ + struct dept_dep *d; + + if (DEPT_WARN_ON(!valid_class(e->class))) + return NULL; + + if (DEPT_WARN_ON(!valid_class(w->class))) + return NULL; + + if (lookup_dep(e->class, w->class)) + return NULL; + + d =3D new_dep(); + if (unlikely(!d)) + return NULL; + + d->ecxt =3D get_ecxt(e); + d->wait =3D get_wait(w); + + /* + * Add the dependency into hash and graph. + */ + hash_add_dep(d); + list_add(&d->dep_node, &dep_fc(d)->dep_head); + list_add(&d->dep_rev_node, &dep_tc(d)->dep_rev_head); + return d; +} + +static void bfs_init_check_dl(void *node, void *in, void **out) +{ + struct dept_class *root =3D (struct dept_class *)node; + struct dept_dep *new =3D (struct dept_dep *)in; + + root->bfs_gen =3D bfs_gen; + dep_tc(new)->bfs_parent =3D dep_fc(new); +} + +static void bfs_extend_dep(struct list_head *h, void *node) +{ + struct dept_class *cur =3D (struct dept_class *)node; + struct dept_dep *d; + + list_for_each_entry(d, &cur->dep_head, dep_node) { + struct dept_class *next =3D dep_tc(d); + + if (bfs_gen =3D=3D next->bfs_gen) + continue; + next->bfs_parent =3D cur; + next->bfs_gen =3D bfs_gen; + list_add_tail(&next->bfs_node, h); + } +} + +static void *bfs_dequeue_dep(struct list_head *h) +{ + struct dept_class *c; + + DEPT_WARN_ON(list_empty(h)); + + c =3D list_first_entry(h, struct dept_class, bfs_node); + list_del(&c->bfs_node); + return c; +} + +static enum bfs_ret cb_check_dl(void *node, void *in, void **out) +{ + struct dept_class *cur =3D (struct dept_class *)node; + struct dept_dep *new =3D (struct dept_dep *)in; + + if (cur =3D=3D dep_fc(new)) { + print_circle(dep_tc(new)); + return BFS_DONE; + } + + return BFS_CONTINUE; +} + +/* + * This function is actually in charge of reporting. + */ +static void check_dl_bfs(struct dept_dep *d) +{ + struct bfs_ops ops =3D { + .bfs_init =3D bfs_init_check_dl, + .extend =3D bfs_extend_dep, + .dequeue =3D bfs_dequeue_dep, + .callback =3D cb_check_dl, + }; + + bfs((void *)dep_tc(d), &ops, (void *)d, NULL); +} + +static void bfs_init_dep(void *node, void *in, void **out) +{ + struct dept_class *root =3D (struct dept_class *)node; + + root->bfs_gen =3D bfs_gen; +} + +static void bfs_extend_dep_rev(struct list_head *h, void *node) +{ + struct dept_class *cur =3D (struct dept_class *)node; + struct dept_dep *d; + + list_for_each_entry(d, &cur->dep_rev_head, dep_rev_node) { + struct dept_class *next =3D dep_fc(d); + + if (bfs_gen =3D=3D next->bfs_gen) + continue; + next->bfs_parent =3D cur; + next->bfs_gen =3D bfs_gen; + list_add_tail(&next->bfs_node, h); + } +} + +static enum bfs_ret cb_find_iw(void *node, void *in, void **out) +{ + struct dept_class *cur =3D (struct dept_class *)node; + int irq =3D *(int *)in; + struct dept_iwait *iw; + + if (DEPT_WARN_ON(!out)) + return BFS_DONE; + + iw =3D iwait(cur, irq); + + /* + * If any parent's ->wait was set, then the children would've + * been touched. + */ + if (!iw->touched) + return BFS_SKIP; + + if (!iw->wait) + return BFS_CONTINUE; + + *out =3D iw; + return BFS_DONE; +} + +static struct dept_iwait *find_iw_bfs(struct dept_class *c, int irq) +{ + struct dept_iwait *iw =3D iwait(c, irq); + struct dept_iwait *found =3D NULL; + struct bfs_ops ops =3D { + .bfs_init =3D bfs_init_dep, + .extend =3D bfs_extend_dep_rev, + .dequeue =3D bfs_dequeue_dep, + .callback =3D cb_find_iw, + }; + + bfs((void *)c, &ops, (void *)&irq, (void **)&found); + + if (found) + return found; + + untouch_iwait(iw); + return NULL; +} + +static enum bfs_ret cb_touch_iw_find_ie(void *node, void *in, void **out) +{ + struct dept_class *cur =3D (struct dept_class *)node; + int irq =3D *(int *)in; + struct dept_iecxt *ie =3D iecxt(cur, irq); + struct dept_iwait *iw =3D iwait(cur, irq); + + if (DEPT_WARN_ON(!out)) + return BFS_DONE; + + touch_iwait(iw); + + if (!ie->ecxt) + return BFS_CONTINUE; + if (!*out) + *out =3D ie; + + /* + * Do touch_iwait() all the way. + */ + return BFS_CONTINUE; +} + +static struct dept_iecxt *touch_iw_find_ie_bfs(struct dept_class *c, + int irq) +{ + struct dept_iecxt *found =3D NULL; + struct bfs_ops ops =3D { + .bfs_init =3D bfs_init_dep, + .extend =3D bfs_extend_dep, + .dequeue =3D bfs_dequeue_dep, + .callback =3D cb_touch_iw_find_ie, + }; + + bfs((void *)c, &ops, (void *)&irq, (void **)&found); + return found; +} + +/* + * Should be called with dept_lock held. + */ +static void __add_idep(struct dept_iecxt *ie, struct dept_iwait *iw) +{ + struct dept_dep *new; + + /* + * There's nothing to do. + */ + if (!ie || !iw || !ie->ecxt || !iw->wait) + return; + + new =3D __add_dep(ie->ecxt, iw->wait); + + /* + * Deadlock detected. Let check_dl_bfs() report it. + */ + if (new) { + check_dl_bfs(new); + stale_iecxt(ie); + stale_iwait(iw); + } + + /* + * If !new, it would be the case of lack of object resource. + * Just let it go and get checked by other chances. Retrying is + * meaningless in that case. + */ +} + +static void set_check_iecxt(struct dept_class *c, int irq, + struct dept_ecxt *e) +{ + struct dept_iecxt *ie =3D iecxt(c, irq); + + set_iecxt(ie, e); + __add_idep(ie, find_iw_bfs(c, irq)); +} + +static void set_check_iwait(struct dept_class *c, int irq, + struct dept_wait *w) +{ + struct dept_iwait *iw =3D iwait(c, irq); + + set_iwait(iw, w); + __add_idep(touch_iw_find_ie_bfs(c, irq), iw); +} + +static void add_iecxt(struct dept_class *c, int irq, struct dept_ecxt *e, + bool stack) +{ + /* + * This access is safe since we ensure e->class has set locally. + */ + struct dept_task *dt =3D dept_task(); + struct dept_iecxt *ie =3D iecxt(c, irq); + + if (DEPT_WARN_ON(!valid_class(c))) + return; + + if (unlikely(READ_ONCE(ie->staled))) + return; + + /* + * Skip add_iecxt() if ie->ecxt has ever been set at least once. + * Which means it has a valid ->ecxt or been staled. + */ + if (READ_ONCE(ie->ecxt)) + return; + + if (unlikely(!dept_lock())) + return; + + if (unlikely(ie->staled)) + goto unlock; + if (ie->ecxt) + goto unlock; + + e->enirqf |=3D (1UL << irq); + + /* + * Should be NULL since it's the first time that these + * enirq_{ip,stack}[irq] have ever set. + */ + DEPT_WARN_ON(e->enirq_ip[irq]); + DEPT_WARN_ON(e->enirq_stack[irq]); + + e->enirq_ip[irq] =3D dt->enirq_ip[irq]; + e->enirq_stack[irq] =3D stack ? get_current_stack() : NULL; + + set_check_iecxt(c, irq, e); +unlock: + dept_unlock(); +} + +static void add_iwait(struct dept_class *c, int irq, struct dept_wait *w) +{ + struct dept_iwait *iw =3D iwait(c, irq); + + if (DEPT_WARN_ON(!valid_class(c))) + return; + + if (unlikely(READ_ONCE(iw->staled))) + return; + + /* + * Skip add_iwait() if iw->wait has ever been set at least once. + * Which means it has a valid ->wait or been staled. + */ + if (READ_ONCE(iw->wait)) + return; + + if (unlikely(!dept_lock())) + return; + + if (unlikely(iw->staled)) + goto unlock; + if (iw->wait) + goto unlock; + + w->irqf |=3D (1UL << irq); + + /* + * Should be NULL since it's the first time that these + * irq_{ip,stack}[irq] have ever set. + */ + DEPT_WARN_ON(w->irq_ip[irq]); + DEPT_WARN_ON(w->irq_stack[irq]); + + w->irq_ip[irq] =3D w->wait_ip; + w->irq_stack[irq] =3D get_current_stack(); + + set_check_iwait(c, irq, w); +unlock: + dept_unlock(); +} + +static struct dept_wait_hist *hist(int pos) +{ + struct dept_task *dt =3D dept_task(); + + return dt->wait_hist + (pos % DEPT_MAX_WAIT_HIST); +} + +static int hist_pos_next(void) +{ + struct dept_task *dt =3D dept_task(); + + return dt->wait_hist_pos % DEPT_MAX_WAIT_HIST; +} + +static void hist_advance(void) +{ + struct dept_task *dt =3D dept_task(); + + dt->wait_hist_pos++; + dt->wait_hist_pos %=3D DEPT_MAX_WAIT_HIST; +} + +static struct dept_wait_hist *new_hist(void) +{ + struct dept_wait_hist *wh =3D hist(hist_pos_next()); + + hist_advance(); + return wh; +} + +static void add_hist(struct dept_wait *w, unsigned int wg, unsigned int ct= xt_id) +{ + struct dept_wait_hist *wh =3D new_hist(); + + if (likely(wh->wait)) + put_wait(wh->wait); + + wh->wait =3D get_wait(w); + wh->wgen =3D wg; + wh->ctxt_id =3D ctxt_id; +} + +/* + * Should be called after setting up e's iecxt and w's iwait. + */ +static void add_dep(struct dept_ecxt *e, struct dept_wait *w) +{ + struct dept_class *fc =3D e->class; + struct dept_class *tc =3D w->class; + struct dept_dep *d; + int i; + + if (lookup_dep(fc, tc)) + return; + + if (unlikely(!dept_lock())) + return; + + /* + * __add_dep() will lookup_dep() again with lock held. + */ + d =3D __add_dep(e, w); + if (d) { + check_dl_bfs(d); + + for (i =3D 0; i < DEPT_IRQS_NR; i++) { + struct dept_iwait *fiw =3D iwait(fc, i); + struct dept_iecxt *found_ie; + struct dept_iwait *found_iw; + + /* + * '->touched =3D=3D false' guarantees there's no + * parent that has been set ->wait. + */ + if (!fiw->touched) + continue; + + /* + * find_iw_bfs() will untouch the iwait if + * not found. + */ + found_iw =3D find_iw_bfs(fc, i); + + if (!found_iw) + continue; + + found_ie =3D touch_iw_find_ie_bfs(tc, i); + __add_idep(found_ie, found_iw); + } + } + dept_unlock(); +} + +static atomic_t wgen =3D ATOMIC_INIT(1); + +static int next_wgen(void) +{ + /* + * Avoid zero wgen. + */ + return atomic_inc_return(&wgen) ?: atomic_inc_return(&wgen); +} + +static void add_wait(struct dept_class *c, unsigned long ip, + const char *w_fn, int sub_l, bool sched_sleep) +{ + struct dept_task *dt =3D dept_task(); + struct dept_wait *w; + unsigned int wg; + int irq; + int i; + + if (DEPT_WARN_ON(!valid_class(c))) + return; + + w =3D new_wait(); + if (unlikely(!w)) + return; + + WRITE_ONCE(w->class, get_class(c)); + w->wait_ip =3D ip; + w->wait_fn =3D w_fn; + w->wait_stack =3D get_current_stack(); + w->sched_sleep =3D sched_sleep; + + irq =3D cur_irq(); + if (irq < DEPT_IRQS_NR) + add_iwait(c, irq, w); + + /* + * Avoid adding dependency between user aware nested ecxt and + * wait. + */ + for (i =3D dt->ecxt_held_pos - 1; i >=3D 0; i--) { + struct dept_ecxt_held *eh; + + eh =3D dt->ecxt_held + i; + + /* + * the case of invalid key'ed one + */ + if (!eh->ecxt) + continue; + + if (eh->ecxt->class !=3D c || eh->sub_l =3D=3D sub_l) + add_dep(eh->ecxt, w); + } + + wg =3D next_wgen(); + add_hist(w, wg, cur_ctxt_id()); + + del_wait(w); +} + +static struct dept_ecxt_held *add_ecxt(struct dept_map *m, + struct dept_class *c, unsigned long ip, const char *c_fn, + const char *e_fn, int sub_l) +{ + struct dept_task *dt =3D dept_task(); + struct dept_ecxt_held *eh; + struct dept_ecxt *e; + unsigned long irqf; + unsigned int wg; + int irq; + + if (DEPT_WARN_ON(!valid_class(c))) + return NULL; + + if (DEPT_WARN_ON_ONCE(dt->ecxt_held_pos >=3D DEPT_MAX_ECXT_HELD)) + return NULL; + + wg =3D next_wgen(); + if (m->nocheck) { + eh =3D dt->ecxt_held + (dt->ecxt_held_pos++); + eh->ecxt =3D NULL; + eh->map =3D m; + eh->class =3D get_class(c); + eh->wgen =3D wg; + eh->sub_l =3D sub_l; + + return eh; + } + + e =3D new_ecxt(); + if (unlikely(!e)) + return NULL; + + e->class =3D get_class(c); + e->ecxt_ip =3D ip; + e->ecxt_stack =3D ip ? get_current_stack() : NULL; + e->event_fn =3D e_fn; + e->ecxt_fn =3D c_fn; + + eh =3D dt->ecxt_held + (dt->ecxt_held_pos++); + eh->ecxt =3D get_ecxt(e); + eh->map =3D m; + eh->class =3D get_class(c); + eh->wgen =3D wg; + eh->sub_l =3D sub_l; + + irqf =3D cur_enirqf(); + for_each_set_bit(irq, &irqf, DEPT_IRQS_NR) + add_iecxt(c, irq, e, false); + + del_ecxt(e); + return eh; +} + +static int find_ecxt_pos(struct dept_map *m, struct dept_class *c, + bool newfirst) +{ + struct dept_task *dt =3D dept_task(); + int i; + + if (newfirst) { + for (i =3D dt->ecxt_held_pos - 1; i >=3D 0; i--) { + struct dept_ecxt_held *eh; + + eh =3D dt->ecxt_held + i; + if (eh->map =3D=3D m && eh->class =3D=3D c) + return i; + } + } else { + for (i =3D 0; i < dt->ecxt_held_pos; i++) { + struct dept_ecxt_held *eh; + + eh =3D dt->ecxt_held + i; + if (eh->map =3D=3D m && eh->class =3D=3D c) + return i; + } + } + return -1; +} + +static bool pop_ecxt(struct dept_map *m, struct dept_class *c) +{ + struct dept_task *dt =3D dept_task(); + int pos; + int i; + + pos =3D find_ecxt_pos(m, c, true); + if (pos =3D=3D -1) + return false; + + if (dt->ecxt_held[pos].class) + put_class(dt->ecxt_held[pos].class); + + if (dt->ecxt_held[pos].ecxt) + put_ecxt(dt->ecxt_held[pos].ecxt); + + dt->ecxt_held_pos--; + + for (i =3D pos; i < dt->ecxt_held_pos; i++) + dt->ecxt_held[i] =3D dt->ecxt_held[i + 1]; + return true; +} + +static bool good_hist(struct dept_wait_hist *wh, unsigned int wg) +{ + return wh->wait !=3D NULL && before(wg, wh->wgen); +} + +/* + * Binary-search the ring buffer for the earliest valid wait. + */ +static int find_hist_pos(unsigned int wg) +{ + int oldest; + int l; + int r; + int pos; + + oldest =3D hist_pos_next(); + if (unlikely(good_hist(hist(oldest), wg))) { + DEPT_INFO_ONCE("Need to expand the ring buffer.\n"); + return oldest; + } + + l =3D oldest + 1; + r =3D oldest + DEPT_MAX_WAIT_HIST - 1; + for (pos =3D (l + r) / 2; l <=3D r; pos =3D (l + r) / 2) { + struct dept_wait_hist *p =3D hist(pos - 1); + struct dept_wait_hist *wh =3D hist(pos); + + if (!good_hist(p, wg) && good_hist(wh, wg)) + return pos % DEPT_MAX_WAIT_HIST; + if (good_hist(wh, wg)) + r =3D pos - 1; + else + l =3D pos + 1; + } + return -1; +} + +static void do_event(struct dept_map *m, struct dept_map *real_m, + struct dept_class *c, unsigned int wg, unsigned long ip, + const char *e_fn) +{ + struct dept_task *dt =3D dept_task(); + struct dept_wait_hist *wh; + struct dept_ecxt_held *eh; + unsigned int ctxt_id; + int end; + int pos; + int i; + + if (DEPT_WARN_ON(!valid_class(c))) + return; + + if (m->nocheck) + return; + + /* + * The event was triggered before wait. + */ + if (!wg) + return; + + /* + * If an ecxt for this map exists, let the ecxt work for this + * event and do not proceed it in do_event(). + */ + if (find_ecxt_pos(real_m, c, false) !=3D -1) + return; + eh =3D add_ecxt(m, c, 0UL, NULL, e_fn, 0); + + if (!eh) + return; + + if (DEPT_WARN_ON(!eh->ecxt)) + goto out; + + eh->ecxt->event_ip =3D ip; + eh->ecxt->event_stack =3D get_current_stack(); + + pos =3D find_hist_pos(wg); + if (pos =3D=3D -1) + goto out; + + ctxt_id =3D cur_ctxt_id(); + end =3D hist_pos_next(); + end =3D end > pos ? end : end + DEPT_MAX_WAIT_HIST; + for (wh =3D hist(pos); pos < end; wh =3D hist(++pos)) { + if (dt->in_sched && wh->wait->sched_sleep) + continue; + + if (wh->ctxt_id =3D=3D ctxt_id) + add_dep(eh->ecxt, wh->wait); + } + + for (i =3D 0; i < DEPT_IRQS_NR; i++) { + struct dept_ecxt *e; + + if (before(dt->wgen_enirq[i], wg)) + continue; + + e =3D eh->ecxt; + add_iecxt(e->class, i, e, false); + } +out: + /* + * Pop ecxt that temporarily has been added to handle this event. + */ + pop_ecxt(m, c); +} + +static void del_dep_rcu(struct rcu_head *rh) +{ + struct dept_dep *d =3D container_of(rh, struct dept_dep, rh); + + preempt_disable(); + del_dep(d); + preempt_enable(); +} + +/* + * NOTE: Must be called with dept_lock held. + */ +static void disconnect_class(struct dept_class *c) +{ + struct dept_dep *d, *n; + int i; + + list_for_each_entry_safe(d, n, &c->dep_head, dep_node) { + list_del_rcu(&d->dep_node); + list_del_rcu(&d->dep_rev_node); + hash_del_dep(d); + call_rcu(&d->rh, del_dep_rcu); + } + + list_for_each_entry_safe(d, n, &c->dep_rev_head, dep_rev_node) { + list_del_rcu(&d->dep_node); + list_del_rcu(&d->dep_rev_node); + hash_del_dep(d); + call_rcu(&d->rh, del_dep_rcu); + } + + for (i =3D 0; i < DEPT_IRQS_NR; i++) { + stale_iecxt(iecxt(c, i)); + stale_iwait(iwait(c, i)); + } +} + +/* + * Context control + * =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D + * Whether a wait is in {hard,soft}-IRQ context or whether + * {hard,soft}-IRQ has been enabled on the way to an event is very + * important to check dependency. All those things should be tracked. + */ + +static unsigned long cur_enirqf(void) +{ + struct dept_task *dt =3D dept_task(); + int he =3D dt->hardirqs_enabled; + int se =3D dt->softirqs_enabled; + + if (he) + return DEPT_HIRQF | (se ? DEPT_SIRQF : 0UL); + return 0UL; +} + +static int cur_irq(void) +{ + if (lockdep_softirq_context(current)) + return DEPT_SIRQ; + if (lockdep_hardirq_context()) + return DEPT_HIRQ; + return DEPT_IRQS_NR; +} + +static unsigned int cur_ctxt_id(void) +{ + struct dept_task *dt =3D dept_task(); + int irq =3D cur_irq(); + + /* + * Normal process context + */ + if (irq =3D=3D DEPT_IRQS_NR) + return 0U; + + return dt->irq_id[irq] | (1UL << irq); +} + +static void enirq_transition(int irq) +{ + struct dept_task *dt =3D dept_task(); + int i; + + /* + * IRQ can cut in on the way to the event. Used for cross-event + * detection. + * + * wait context event context(ecxt) + * ------------ ------------------- + * wait event + * UPDATE wgen + * observe IRQ enabled + * UPDATE wgen + * keep the wgen locally + * + * on the event + * check the wgen kept + */ + + dt->wgen_enirq[irq] =3D next_wgen(); + + for (i =3D dt->ecxt_held_pos - 1; i >=3D 0; i--) { + struct dept_ecxt_held *eh; + struct dept_ecxt *e; + + eh =3D dt->ecxt_held + i; + e =3D eh->ecxt; + if (e) + add_iecxt(e->class, irq, e, true); + } +} + +static void dept_enirq(unsigned long ip) +{ + struct dept_task *dt =3D dept_task(); + unsigned long irqf =3D cur_enirqf(); + int irq; + unsigned long flags; + + if (unlikely(!dept_working())) + return; + + /* + * IRQ ON/OFF transition might happen while Dept is working. + * We cannot handle recursive entrance. Just ignore it. + * Only transitions outside of Dept will be considered. + */ + if (dt->recursive) + return; + + flags =3D dept_enter(); + + for_each_set_bit(irq, &irqf, DEPT_IRQS_NR) { + dt->enirq_ip[irq] =3D ip; + enirq_transition(irq); + } + + dept_exit(flags); +} + +void dept_softirqs_on_ip(unsigned long ip) +{ + /* + * Assumes that it's called with IRQ disabled so that accessing + * current's fields is not racy. + */ + dept_task()->softirqs_enabled =3D true; + dept_enirq(ip); +} + +void dept_hardirqs_on(void) +{ + /* + * Assumes that it's called with IRQ disabled so that accessing + * current's fields is not racy. + */ + dept_task()->hardirqs_enabled =3D true; + dept_enirq(_RET_IP_); +} + +void dept_softirqs_off(void) +{ + /* + * Assumes that it's called with IRQ disabled so that accessing + * current's fields is not racy. + */ + dept_task()->softirqs_enabled =3D false; +} + +void dept_hardirqs_off(void) +{ + /* + * Assumes that it's called with IRQ disabled so that accessing + * current's fields is not racy. + */ + dept_task()->hardirqs_enabled =3D false; +} + +/* + * Ensure it's the outmost softirq context. + */ +void dept_softirq_enter(void) +{ + struct dept_task *dt =3D dept_task(); + + dt->irq_id[DEPT_SIRQ] +=3D 1UL << DEPT_IRQS_NR; +} + +/* + * Ensure it's the outmost hardirq context. + */ +void dept_hardirq_enter(void) +{ + struct dept_task *dt =3D dept_task(); + + dt->irq_id[DEPT_HIRQ] +=3D 1UL << DEPT_IRQS_NR; +} + +void dept_sched_enter(void) +{ + dept_task()->in_sched =3D true; +} + +void dept_sched_exit(void) +{ + dept_task()->in_sched =3D false; +} + +/* + * Exposed APIs + * =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D + */ + +static void clean_classes_cache(struct dept_key *k) +{ + int i; + + for (i =3D 0; i < DEPT_MAX_SUBCLASSES_CACHE; i++) { + if (!READ_ONCE(k->classes[i])) + continue; + + WRITE_ONCE(k->classes[i], NULL); + } +} + +/* + * Assume we don't have to consider race with the map when + * dept_map_init() is called. + */ +void dept_map_init(struct dept_map *m, struct dept_key *k, int sub_u, + const char *n) +{ + unsigned long flags; + + if (unlikely(!dept_working())) { + m->nocheck =3D true; + return; + } + + if (DEPT_WARN_ON(sub_u < 0)) { + m->nocheck =3D true; + return; + } + + if (DEPT_WARN_ON(sub_u >=3D DEPT_MAX_SUBCLASSES_USR)) { + m->nocheck =3D true; + return; + } + + /* + * Allow recursive entrance. + */ + flags =3D dept_enter_recursive(); + + clean_classes_cache(&m->map_key); + + m->keys =3D k; + m->sub_u =3D sub_u; + m->name =3D n; + m->wgen =3D 0U; + m->nocheck =3D !valid_key(k); + + dept_exit_recursive(flags); +} +EXPORT_SYMBOL_GPL(dept_map_init); + +/* + * Assume we don't have to consider race with the map when + * dept_map_reinit() is called. + */ +void dept_map_reinit(struct dept_map *m, struct dept_key *k, int sub_u, + const char *n) +{ + unsigned long flags; + + if (unlikely(!dept_working())) { + m->nocheck =3D true; + return; + } + + /* + * Allow recursive entrance. + */ + flags =3D dept_enter_recursive(); + + if (k) { + clean_classes_cache(&m->map_key); + m->keys =3D k; + m->nocheck =3D !valid_key(k); + } + + if (sub_u >=3D 0 && sub_u < DEPT_MAX_SUBCLASSES_USR) + m->sub_u =3D sub_u; + + if (n) + m->name =3D n; + + m->wgen =3D 0U; + + dept_exit_recursive(flags); +} +EXPORT_SYMBOL_GPL(dept_map_reinit); + +void dept_map_copy(struct dept_map *to, struct dept_map *from) +{ + if (unlikely(!dept_working())) { + to->nocheck =3D true; + return; + } + + *to =3D *from; + + /* + * XXX: 'to' might be in a stack or something. Using the address + * in a stack segment as a key is meaningless. Just ignore the + * case for now. + */ + if (!to->keys) { + to->nocheck =3D true; + return; + } + + /* + * Since the class cache can be modified concurrently we could + * observe half pointers (64bit arch using 32bit copy + * instructions). Therefore clear the caches and take the + * performance hit. + */ + clean_classes_cache(&to->map_key); +} + +static LIST_HEAD(classes); + +static bool within(const void *addr, void *start, unsigned long size) +{ + return addr >=3D start && addr < start + size; +} + +void dept_free_range(void *start, unsigned int sz) +{ + struct dept_task *dt =3D dept_task(); + struct dept_class *c, *n; + unsigned long flags; + + if (unlikely(!dept_working())) + return; + + if (dt->recursive) { + DEPT_STOP("Failed to successfully free Dept objects.\n"); + return; + } + + flags =3D dept_enter(); + + /* + * dept_free_range() should not fail. + * + * FIXME: Should be fixed if dept_free_range() causes deadlock + * with dept_lock(). + */ + while (unlikely(!dept_lock())) + cpu_relax(); + + list_for_each_entry_safe(c, n, &classes, all_node) { + if (!within((void *)c->key, start, sz) && + !within(c->name, start, sz)) + continue; + + hash_del_class(c); + disconnect_class(c); + list_del(&c->all_node); + invalidate_class(c); + + /* + * Actual deletion will happen on the rcu callback + * that has been added in disconnect_class(). + */ + del_class(c); + } + dept_unlock(); + dept_exit(flags); + + /* + * Wait until even lockless hash_lookup_class() for the class + * returns NULL. + */ + might_sleep(); + synchronize_rcu(); +} + +static int sub_id(struct dept_map *m, int e) +{ + return (m ? m->sub_u : 0) + e * DEPT_MAX_SUBCLASSES_USR; +} + +static struct dept_class *check_new_class(struct dept_key *local, + struct dept_key *k, int sub_id, + const char *n, bool sched_map) +{ + struct dept_class *c =3D NULL; + + if (DEPT_WARN_ON(sub_id >=3D DEPT_MAX_SUBCLASSES)) + return NULL; + + if (DEPT_WARN_ON(!k)) + return NULL; + + /* + * XXX: Assume that users prevent the map from using if any of + * the cached keys has been invalidated. If not, the cache, + * local->classes should not be used because it would be racy + * with class deletion. + */ + if (local && sub_id < DEPT_MAX_SUBCLASSES_CACHE) + c =3D READ_ONCE(local->classes[sub_id]); + + if (c) + return c; + + c =3D lookup_class((unsigned long)k->base + sub_id); + if (c) + goto caching; + + if (unlikely(!dept_lock())) + return NULL; + + c =3D lookup_class((unsigned long)k->base + sub_id); + if (unlikely(c)) + goto unlock; + + c =3D new_class(); + if (unlikely(!c)) + goto unlock; + + c->name =3D n; + c->sched_map =3D sched_map; + c->sub_id =3D sub_id; + c->key =3D (unsigned long)(k->base + sub_id); + hash_add_class(c); + list_add(&c->all_node, &classes); +unlock: + dept_unlock(); +caching: + if (local && sub_id < DEPT_MAX_SUBCLASSES_CACHE) + WRITE_ONCE(local->classes[sub_id], c); + + return c; +} + +/* + * Called between dept_enter() and dept_exit(). + */ +static void __dept_wait(struct dept_map *m, unsigned long w_f, + unsigned long ip, const char *w_fn, int sub_l, + bool sched_sleep, bool sched_map) +{ + int e; + + /* + * Be as conservative as possible. In case of multiple waits for + * a single dept_map, we are going to keep only the last wait's + * wgen for simplicity - keeping all wgens seems overengineering. + * + * Of course, it might cause missing some dependencies that + * would rarely, probably never, happen but it helps avoid + * false positive reports. + */ + for_each_set_bit(e, &w_f, DEPT_MAX_SUBCLASSES_EVT) { + struct dept_class *c; + struct dept_key *k; + + k =3D m->keys ?: &m->map_key; + c =3D check_new_class(&m->map_key, k, + sub_id(m, e), m->name, sched_map); + if (!c) + continue; + + add_wait(c, ip, w_fn, sub_l, sched_sleep); + } +} + +/* + * Called between dept_enter() and dept_exit(). + */ +static void __dept_event(struct dept_map *m, struct dept_map *real_m, + unsigned long e_f, unsigned long ip, const char *e_fn, + bool sched_map) +{ + struct dept_class *c; + struct dept_key *k; + int e; + + e =3D find_first_bit(&e_f, DEPT_MAX_SUBCLASSES_EVT); + + if (DEPT_WARN_ON(e >=3D DEPT_MAX_SUBCLASSES_EVT)) + return; + + /* + * An event is an event. If the caller passed more than single + * event, then warn it and handle the event corresponding to + * the first bit anyway. + */ + DEPT_WARN_ON(1UL << e !=3D e_f); + + k =3D m->keys ?: &m->map_key; + c =3D check_new_class(&m->map_key, k, sub_id(m, e), m->name, sched_map); + + if (c) + do_event(m, real_m, c, READ_ONCE(m->wgen), ip, e_fn); +} + +void dept_wait(struct dept_map *m, unsigned long w_f, + unsigned long ip, const char *w_fn, int sub_l) +{ + struct dept_task *dt =3D dept_task(); + unsigned long flags; + + if (unlikely(!dept_working())) + return; + + if (dt->recursive) + return; + + if (m->nocheck) + return; + + flags =3D dept_enter(); + + __dept_wait(m, w_f, ip, w_fn, sub_l, false, false); + + dept_exit(flags); +} +EXPORT_SYMBOL_GPL(dept_wait); + +void dept_stage_wait(struct dept_map *m, struct dept_key *k, + unsigned long ip, const char *w_fn) +{ + struct dept_task *dt =3D dept_task(); + unsigned long flags; + + if (unlikely(!dept_working())) + return; + + if (m && m->nocheck) + return; + + /* + * Either m or k should be passed. Which means Dept relies on + * either its own map or the caller's position in the code when + * determining its class. + */ + if (DEPT_WARN_ON(!m && !k)) + return; + + /* + * Allow recursive entrance. + */ + flags =3D dept_enter_recursive(); + + /* + * Ensure the outmost dept_stage_wait() works. + */ + if (dt->stage_m.keys) + goto exit; + + arch_spin_lock(&dt->stage_lock); + if (m) { + dt->stage_m =3D *m; + dt->stage_real_m =3D m; + + /* + * Ensure dt->stage_m.keys !=3D NULL and it works with the + * map's map_key, not stage_m's one when ->keys =3D=3D NULL. + */ + if (!m->keys) + dt->stage_m.keys =3D &m->map_key; + } else { + dt->stage_m.name =3D w_fn; + dt->stage_sched_map =3D true; + dt->stage_real_m =3D &dt->stage_m; + } + + /* + * dept_map_reinit() includes WRITE_ONCE(->wgen, 0U) that + * effectively disables the map just in case real sleep won't + * happen. dept_request_event_wait_commit() will enable it. + */ + dept_map_reinit(&dt->stage_m, k, -1, NULL); + + dt->stage_w_fn =3D w_fn; + dt->stage_ip =3D ip; + arch_spin_unlock(&dt->stage_lock); +exit: + dept_exit_recursive(flags); +} +EXPORT_SYMBOL_GPL(dept_stage_wait); + +static void __dept_clean_stage(struct dept_task *dt) +{ + memset(&dt->stage_m, 0x0, sizeof(struct dept_map)); + dt->stage_real_m =3D NULL; + dt->stage_sched_map =3D false; + dt->stage_w_fn =3D NULL; + dt->stage_ip =3D 0UL; +} + +void dept_clean_stage(void) +{ + struct dept_task *dt =3D dept_task(); + unsigned long flags; + + if (unlikely(!dept_working())) + return; + + /* + * Allow recursive entrance. + */ + flags =3D dept_enter_recursive(); + arch_spin_lock(&dt->stage_lock); + __dept_clean_stage(dt); + arch_spin_unlock(&dt->stage_lock); + dept_exit_recursive(flags); +} +EXPORT_SYMBOL_GPL(dept_clean_stage); + +/* + * Always called from __schedule(). + */ +void dept_request_event_wait_commit(void) +{ + struct dept_task *dt =3D dept_task(); + unsigned long flags; + unsigned int wg; + unsigned long ip; + const char *w_fn; + bool sched_map; + + if (unlikely(!dept_working())) + return; + + /* + * It's impossible that __schedule() is called while Dept is + * working that already disabled IRQ at the entrance. + */ + if (DEPT_WARN_ON(dt->recursive)) + return; + + flags =3D dept_enter(); + + arch_spin_lock(&dt->stage_lock); + + /* + * Checks if current has staged a wait. + */ + if (!dt->stage_m.keys) { + arch_spin_unlock(&dt->stage_lock); + goto exit; + } + + w_fn =3D dt->stage_w_fn; + ip =3D dt->stage_ip; + sched_map =3D dt->stage_sched_map; + + wg =3D next_wgen(); + WRITE_ONCE(dt->stage_m.wgen, wg); + arch_spin_unlock(&dt->stage_lock); + + __dept_wait(&dt->stage_m, 1UL, ip, w_fn, 0, true, sched_map); +exit: + dept_exit(flags); +} + +/* + * Always called from try_to_wake_up(). + */ +void dept_ttwu_stage_wait(struct task_struct *requestor, unsigned long ip) +{ + struct dept_task *dt =3D dept_task(); + struct dept_task *dt_req =3D &requestor->dept_task; + unsigned long flags; + struct dept_map m; + struct dept_map *real_m; + bool sched_map; + + if (unlikely(!dept_working())) + return; + + if (dt->recursive) + return; + + flags =3D dept_enter(); + + arch_spin_lock(&dt_req->stage_lock); + + /* + * Serializing is unnecessary as long as it always comes from + * try_to_wake_up(). + */ + m =3D dt_req->stage_m; + sched_map =3D dt_req->stage_sched_map; + real_m =3D dt_req->stage_real_m; + __dept_clean_stage(dt_req); + arch_spin_unlock(&dt_req->stage_lock); + + /* + * ->stage_m.keys should not be NULL if it's in use. Should + * make sure that it's not NULL when staging a valid map. + */ + if (!m.keys) + goto exit; + + __dept_event(&m, real_m, 1UL, ip, "try_to_wake_up", sched_map); +exit: + dept_exit(flags); +} + +/* + * Modifies the latest ecxt corresponding to m and e_f. + */ +void dept_map_ecxt_modify(struct dept_map *m, unsigned long e_f, + struct dept_key *new_k, unsigned long new_e_f, + unsigned long new_ip, const char *new_c_fn, + const char *new_e_fn, int new_sub_l) +{ + struct dept_task *dt =3D dept_task(); + struct dept_ecxt_held *eh; + struct dept_class *c; + struct dept_key *k; + unsigned long flags; + int pos =3D -1; + int new_e; + int e; + + if (unlikely(!dept_working())) + return; + + /* + * XXX: Couldn't handle re-enterance cases. Ignore it for now. + */ + if (dt->recursive) + return; + + /* + * Should go ahead no matter whether ->nocheck =3D=3D true or not + * because ->nocheck value can be changed within the ecxt area + * delimitated by dept_ecxt_enter() and dept_ecxt_exit(). + */ + + flags =3D dept_enter(); + + for_each_set_bit(e, &e_f, DEPT_MAX_SUBCLASSES_EVT) { + k =3D m->keys ?: &m->map_key; + c =3D check_new_class(&m->map_key, k, + sub_id(m, e), m->name, false); + if (!c) + continue; + + /* + * When it found an ecxt for any event in e_f, done. + */ + pos =3D find_ecxt_pos(m, c, true); + if (pos !=3D -1) + break; + } + + if (unlikely(pos =3D=3D -1)) + goto exit; + + eh =3D dt->ecxt_held + pos; + new_sub_l =3D new_sub_l >=3D 0 ? new_sub_l : eh->sub_l; + + new_e =3D find_first_bit(&new_e_f, DEPT_MAX_SUBCLASSES_EVT); + + if (new_e < DEPT_MAX_SUBCLASSES_EVT) + /* + * Let it work with the first bit anyway. + */ + DEPT_WARN_ON(1UL << new_e !=3D new_e_f); + else + new_e =3D e; + + pop_ecxt(m, c); + + /* + * Apply the key to the map. + */ + if (new_k) + dept_map_reinit(m, new_k, -1, NULL); + + k =3D m->keys ?: &m->map_key; + c =3D check_new_class(&m->map_key, k, sub_id(m, new_e), m->name, false); + + if (c && add_ecxt(m, c, new_ip, new_c_fn, new_e_fn, new_sub_l)) + goto exit; + + /* + * Successfully pop_ecxt()ed but failed to add_ecxt(). + */ + dt->missing_ecxt++; +exit: + dept_exit(flags); +} +EXPORT_SYMBOL_GPL(dept_map_ecxt_modify); + +void dept_ecxt_enter(struct dept_map *m, unsigned long e_f, unsigned long = ip, + const char *c_fn, const char *e_fn, int sub_l) +{ + struct dept_task *dt =3D dept_task(); + unsigned long flags; + struct dept_class *c; + struct dept_key *k; + int e; + + if (unlikely(!dept_working())) + return; + + if (dt->recursive) { + dt->missing_ecxt++; + return; + } + + /* + * Should go ahead no matter whether ->nocheck =3D=3D true or not + * because ->nocheck value can be changed within the ecxt area + * delimitated by dept_ecxt_enter() and dept_ecxt_exit(). + */ + + flags =3D dept_enter(); + + e =3D find_first_bit(&e_f, DEPT_MAX_SUBCLASSES_EVT); + + if (e >=3D DEPT_MAX_SUBCLASSES_EVT) + goto missing_ecxt; + + /* + * An event is an event. If the caller passed more than single + * event, then warn it and handle the event corresponding to + * the first bit anyway. + */ + DEPT_WARN_ON(1UL << e !=3D e_f); + + k =3D m->keys ?: &m->map_key; + c =3D check_new_class(&m->map_key, k, sub_id(m, e), m->name, false); + + if (c && add_ecxt(m, c, ip, c_fn, e_fn, sub_l)) + goto exit; +missing_ecxt: + dt->missing_ecxt++; +exit: + dept_exit(flags); +} +EXPORT_SYMBOL_GPL(dept_ecxt_enter); + +bool dept_ecxt_holding(struct dept_map *m, unsigned long e_f) +{ + struct dept_task *dt =3D dept_task(); + unsigned long flags; + bool ret =3D false; + int e; + + if (unlikely(!dept_working())) + return false; + + if (dt->recursive) + return false; + + flags =3D dept_enter(); + + for_each_set_bit(e, &e_f, DEPT_MAX_SUBCLASSES_EVT) { + struct dept_class *c; + struct dept_key *k; + + k =3D m->keys ?: &m->map_key; + c =3D check_new_class(&m->map_key, k, + sub_id(m, e), m->name, false); + if (!c) + continue; + + if (find_ecxt_pos(m, c, true) !=3D -1) { + ret =3D true; + break; + } + } + + dept_exit(flags); + + return ret; +} +EXPORT_SYMBOL_GPL(dept_ecxt_holding); + +void dept_request_event(struct dept_map *m) +{ + unsigned long flags; + unsigned int wg; + + if (unlikely(!dept_working())) + return; + + if (m->nocheck) + return; + + /* + * Allow recursive entrance. + */ + flags =3D dept_enter_recursive(); + + wg =3D next_wgen(); + WRITE_ONCE(m->wgen, wg); + + dept_exit_recursive(flags); +} +EXPORT_SYMBOL_GPL(dept_request_event); + +void dept_event(struct dept_map *m, unsigned long e_f, + unsigned long ip, const char *e_fn) +{ + struct dept_task *dt =3D dept_task(); + unsigned long flags; + + if (unlikely(!dept_working())) + return; + + if (m->nocheck) + return; + + if (dt->recursive) { + /* + * Dept won't work with this even though an event + * context has been asked. Don't make it confused at + * handling the event. Disable it until the next. + */ + WRITE_ONCE(m->wgen, 0U); + return; + } + + flags =3D dept_enter(); + + __dept_event(m, m, e_f, ip, e_fn, false); + + /* + * Keep the map diabled until the next sleep. + */ + WRITE_ONCE(m->wgen, 0U); + + dept_exit(flags); +} +EXPORT_SYMBOL_GPL(dept_event); + +void dept_ecxt_exit(struct dept_map *m, unsigned long e_f, + unsigned long ip) +{ + struct dept_task *dt =3D dept_task(); + unsigned long flags; + int e; + + if (unlikely(!dept_working())) + return; + + if (dt->recursive) { + dt->missing_ecxt--; + return; + } + + /* + * Should go ahead no matter whether ->nocheck =3D=3D true or not + * because ->nocheck value can be changed within the ecxt area + * delimitated by dept_ecxt_enter() and dept_ecxt_exit(). + */ + + flags =3D dept_enter(); + + for_each_set_bit(e, &e_f, DEPT_MAX_SUBCLASSES_EVT) { + struct dept_class *c; + struct dept_key *k; + + k =3D m->keys ?: &m->map_key; + c =3D check_new_class(&m->map_key, k, + sub_id(m, e), m->name, false); + if (!c) + continue; + + /* + * When it found an ecxt for any event in e_f, done. + */ + if (pop_ecxt(m, c)) + goto exit; + } + + dt->missing_ecxt--; +exit: + dept_exit(flags); +} +EXPORT_SYMBOL_GPL(dept_ecxt_exit); + +void dept_task_exit(struct task_struct *t) +{ + struct dept_task *dt =3D &t->dept_task; + int i; + + if (unlikely(!dept_working())) + return; + + raw_local_irq_disable(); + + if (dt->stack) { + put_stack(dt->stack); + dt->stack =3D NULL; + } + + for (i =3D 0; i < dt->ecxt_held_pos; i++) { + if (dt->ecxt_held[i].class) { + put_class(dt->ecxt_held[i].class); + dt->ecxt_held[i].class =3D NULL; + } + if (dt->ecxt_held[i].ecxt) { + put_ecxt(dt->ecxt_held[i].ecxt); + dt->ecxt_held[i].ecxt =3D NULL; + } + } + + for (i =3D 0; i < DEPT_MAX_WAIT_HIST; i++) { + if (dt->wait_hist[i].wait) { + put_wait(dt->wait_hist[i].wait); + dt->wait_hist[i].wait =3D NULL; + } + } + + dt->task_exit =3D true; + dept_off(); + + raw_local_irq_enable(); +} + +void dept_task_init(struct task_struct *t) +{ + memset(&t->dept_task, 0x0, sizeof(struct dept_task)); + t->dept_task.stage_lock =3D (arch_spinlock_t)__ARCH_SPIN_LOCK_UNLOCKED; +} + +void dept_key_init(struct dept_key *k) +{ + struct dept_task *dt =3D dept_task(); + unsigned long flags; + int sub_id; + + if (unlikely(!dept_working())) + return; + + if (dt->recursive) { + DEPT_STOP("Key initialization fails.\n"); + return; + } + + flags =3D dept_enter(); + + clean_classes_cache(k); + + /* + * dept_key_init() should not fail. + * + * FIXME: Should be fixed if dept_key_init() causes deadlock + * with dept_lock(). + */ + while (unlikely(!dept_lock())) + cpu_relax(); + + for (sub_id =3D 0; sub_id < DEPT_MAX_SUBCLASSES; sub_id++) { + struct dept_class *c; + + c =3D lookup_class((unsigned long)k->base + sub_id); + if (!c) + continue; + + DEPT_STOP("The class(%s/%d) has not been removed.\n", + c->name, sub_id); + break; + } + + dept_unlock(); + dept_exit(flags); +} +EXPORT_SYMBOL_GPL(dept_key_init); + +void dept_key_destroy(struct dept_key *k) +{ + struct dept_task *dt =3D dept_task(); + unsigned long flags; + int sub_id; + + if (unlikely(!dept_working())) + return; + + if (dt->recursive =3D=3D 1 && dt->task_exit) { + /* + * Need to allow to go ahead in this case where + * ->recursive has been set to 1 by dept_off() in + * dept_task_exit() and ->task_exit has been set to + * true in dept_task_exit(). + */ + } else if (dt->recursive) { + DEPT_STOP("Key destroying fails.\n"); + return; + } + + flags =3D dept_enter(); + + /* + * dept_key_destroy() should not fail. + * + * FIXME: Should be fixed if dept_key_destroy() causes deadlock + * with dept_lock(). + */ + while (unlikely(!dept_lock())) + cpu_relax(); + + for (sub_id =3D 0; sub_id < DEPT_MAX_SUBCLASSES; sub_id++) { + struct dept_class *c; + + c =3D lookup_class((unsigned long)k->base + sub_id); + if (!c) + continue; + + hash_del_class(c); + disconnect_class(c); + list_del(&c->all_node); + invalidate_class(c); + + /* + * Actual deletion will happen on the rcu callback + * that has been added in disconnect_class(). + */ + del_class(c); + } + + dept_unlock(); + dept_exit(flags); + + /* + * Wait until even lockless hash_lookup_class() for the class + * returns NULL. + */ + might_sleep(); + synchronize_rcu(); +} +EXPORT_SYMBOL_GPL(dept_key_destroy); + +static void move_llist(struct llist_head *to, struct llist_head *from) +{ + struct llist_node *first =3D llist_del_all(from); + struct llist_node *last; + + if (!first) + return; + + for (last =3D first; last->next; last =3D last->next); + llist_add_batch(first, last, to); +} + +static void migrate_per_cpu_pool(void) +{ + const int boot_cpu =3D 0; + int i; + + /* + * The boot CPU has been using the temporal local pool so far. + * From now on that per_cpu areas have been ready, use the + * per_cpu local pool instead. + */ + DEPT_WARN_ON(smp_processor_id() !=3D boot_cpu); + for (i =3D 0; i < OBJECT_NR; i++) { + struct llist_head *from; + struct llist_head *to; + + from =3D &pool[i].boot_pool; + to =3D per_cpu_ptr(pool[i].lpool, boot_cpu); + move_llist(to, from); + } +} + +#define B2KB(B) ((B) / 1024) + +/* + * Should be called after setup_per_cpu_areas() and before no non-boot + * CPUs have been on. + */ +void __init dept_init(void) +{ + size_t mem_total =3D 0; + + local_irq_disable(); + dept_per_cpu_ready =3D 1; + migrate_per_cpu_pool(); + local_irq_enable(); + +#define HASH(id, bits) BUILD_BUG_ON(1 << (bits) <=3D 0); + #include "dept_hash.h" +#undef HASH +#define OBJECT(id, nr) mem_total +=3D sizeof(struct dept_##id) * nr; + #include "dept_object.h" +#undef OBJECT +#define HASH(id, bits) mem_total +=3D sizeof(struct hlist_head) * (1 << (b= its)); + #include "dept_hash.h" +#undef HASH + + pr_info("DEPendency Tracker: Copyright (c) 2020 LG Electronics, Inc., Byu= ngchul Park\n"); + pr_info("... DEPT_MAX_STACK_ENTRY: %d\n", DEPT_MAX_STACK_ENTRY); + pr_info("... DEPT_MAX_WAIT_HIST : %d\n", DEPT_MAX_WAIT_HIST); + pr_info("... DEPT_MAX_ECXT_HELD : %d\n", DEPT_MAX_ECXT_HELD); + pr_info("... DEPT_MAX_SUBCLASSES : %d\n", DEPT_MAX_SUBCLASSES); +#define OBJECT(id, nr) \ + pr_info("... memory used by %s: %zu KB\n", \ + #id, B2KB(sizeof(struct dept_##id) * nr)); + #include "dept_object.h" +#undef OBJECT +#define HASH(id, bits) \ + pr_info("... hash list head used by %s: %zu KB\n", \ + #id, B2KB(sizeof(struct hlist_head) * (1 << (bits)))); + #include "dept_hash.h" +#undef HASH + pr_info("... total memory used by objects and hashs: %zu KB\n", B2KB(mem_= total)); + pr_info("... per task memory footprint: %zu bytes\n", sizeof(struct dept_= task)); +} diff --git a/kernel/dependency/dept_hash.h b/kernel/dependency/dept_hash.h new file mode 100644 index 000000000000..fd85aab1fdfb --- /dev/null +++ b/kernel/dependency/dept_hash.h @@ -0,0 +1,10 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* + * HASH(id, bits) + * + * id : Id for the object of struct dept_##id. + * bits: 1UL << bits is the hash table size. + */ + +HASH(dep, 12) +HASH(class, 12) diff --git a/kernel/dependency/dept_object.h b/kernel/dependency/dept_objec= t.h new file mode 100644 index 000000000000..0b7eb16fe9fb --- /dev/null +++ b/kernel/dependency/dept_object.h @@ -0,0 +1,13 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* + * OBJECT(id, nr) + * + * id: Id for the object of struct dept_##id. + * nr: # of the object that should be kept in the pool. + */ + +OBJECT(dep, 1024 * 8) +OBJECT(class, 1024 * 8) +OBJECT(stack, 1024 * 32) +OBJECT(ecxt, 1024 * 16) +OBJECT(wait, 1024 * 32) diff --git a/kernel/exit.c b/kernel/exit.c index 1b51dc099f1e..5596de5a8349 100644 --- a/kernel/exit.c +++ b/kernel/exit.c @@ -1001,6 +1001,7 @@ void __noreturn do_exit(long code) exit_tasks_rcu_finish(); =20 lockdep_free_task(tsk); + dept_task_exit(tsk); do_task_dead(); } =20 diff --git a/kernel/fork.c b/kernel/fork.c index c4b26cd8998b..9630060a5005 100644 --- a/kernel/fork.c +++ b/kernel/fork.c @@ -105,6 +105,7 @@ #include #include #include +#include =20 #include #include @@ -2382,6 +2383,7 @@ __latent_entropy struct task_struct *copy_process( #ifdef CONFIG_LOCKDEP lockdep_init_task(p); #endif + dept_task_init(p); =20 #ifdef CONFIG_DEBUG_MUTEXES p->blocked_on =3D NULL; /* not blocked yet */ diff --git a/kernel/module/main.c b/kernel/module/main.c index a2859dc3eea6..7e569e1b4db5 100644 --- a/kernel/module/main.c +++ b/kernel/module/main.c @@ -1286,12 +1286,14 @@ static void free_mod_mem(struct module *mod) =20 /* Free lock-classes; relies on the preceding sync_rcu(). */ lockdep_free_key_range(mod_mem->base, mod_mem->size); + dept_free_range(mod_mem->base, mod_mem->size); if (mod_mem->size) module_memory_free(mod, type); } =20 /* MOD_DATA hosts mod, so free it at last */ lockdep_free_key_range(mod->mem[MOD_DATA].base, mod->mem[MOD_DATA].size); + dept_free_range(mod->mem[MOD_DATA].base, mod->mem[MOD_DATA].size); module_memory_free(mod, MOD_DATA); } =20 @@ -3448,6 +3450,8 @@ static int load_module(struct load_info *info, const = char __user *uargs, for_class_mod_mem_type(type, core_data) { lockdep_free_key_range(mod->mem[type].base, mod->mem[type].size); + dept_free_range(mod->mem[type].base, + mod->mem[type].size); } =20 module_memory_restore_rox(mod); diff --git a/kernel/sched/core.c b/kernel/sched/core.c index c81cf642dba0..d5c90fa93776 100644 --- a/kernel/sched/core.c +++ b/kernel/sched/core.c @@ -66,6 +66,7 @@ #include #include #include +#include =20 #ifdef CONFIG_PREEMPT_DYNAMIC # ifdef CONFIG_GENERIC_ENTRY @@ -4242,6 +4243,7 @@ int try_to_wake_up(struct task_struct *p, unsigned in= t state, int wake_flags) if (READ_ONCE(p->on_rq) && ttwu_runnable(p, wake_flags)) break; =20 + dept_ttwu_stage_wait(p, _RET_IP_); #ifdef CONFIG_SMP /* * Ensure we load p->on_cpu _after_ p->on_rq, otherwise it would be @@ -6663,6 +6665,11 @@ static void __sched notrace __schedule(int sched_mod= e) rq =3D cpu_rq(cpu); prev =3D rq->curr; =20 + prev_state =3D READ_ONCE(prev->__state); + if (sched_mode !=3D SM_PREEMPT && prev_state & TASK_NORMAL) + dept_request_event_wait_commit(); + + dept_sched_enter(); schedule_debug(prev, preempt); =20 if (sched_feat(HRTICK) || sched_feat(HRTICK_DL)) @@ -6771,6 +6778,7 @@ static void __sched notrace __schedule(int sched_mode) raw_spin_rq_unlock_irq(rq); } trace_sched_exit_tp(is_switch, CALLER_ADDR0); + dept_sched_exit(); } =20 void __noreturn do_task_dead(void) diff --git a/lib/Kconfig.debug b/lib/Kconfig.debug index f9051ab610d5..54b03db59a12 100644 --- a/lib/Kconfig.debug +++ b/lib/Kconfig.debug @@ -1365,6 +1365,32 @@ config DEBUG_PREEMPT =20 menu "Lock Debugging (spinlocks, mutexes, etc...)" =20 +config DEPT + bool "Dependency tracking (EXPERIMENTAL)" + depends on DEBUG_KERNEL && LOCK_DEBUGGING_SUPPORT + select DEBUG_SPINLOCK + select DEBUG_MUTEXES if !PREEMPT_RT + select DEBUG_RT_MUTEXES if RT_MUTEXES + select DEBUG_RWSEMS if !PREEMPT_RT + select DEBUG_WW_MUTEX_SLOWPATH + select DEBUG_LOCK_ALLOC + select TRACE_IRQFLAGS + select STACKTRACE + select KALLSYMS + select KALLSYMS_ALL + select PROVE_LOCKING + default n + help + Check dependencies between wait and event and report it if + deadlock possibility has been detected. Multiple reports are + allowed if there are more than a single problem. + + This feature is considered EXPERIMENTAL that might produce + false positive reports because new dependencies start to be + tracked, that have never been tracked before. It's worth + noting, to mitigate the impact by the false positives, multi + reporting has been supported. + config LOCK_DEBUGGING_SUPPORT bool depends on TRACE_IRQFLAGS_SUPPORT && STACKTRACE_SUPPORT && LOCKDEP_SUPPORT diff --git a/lib/locking-selftest.c b/lib/locking-selftest.c index ed99344317f5..18228afccea5 100644 --- a/lib/locking-selftest.c +++ b/lib/locking-selftest.c @@ -1398,6 +1398,8 @@ static void reset_locks(void) local_irq_disable(); lockdep_free_key_range(&ww_lockdep.acquire_key, 1); lockdep_free_key_range(&ww_lockdep.mutex_key, 1); + dept_free_range(&ww_lockdep.acquire_key, 1); + dept_free_range(&ww_lockdep.mutex_key, 1); =20 I1(A); I1(B); I1(C); I1(D); I1(X1); I1(X2); I1(Y1); I1(Y2); I1(Z1); I1(Z2); --=20 2.17.1 From nobody Fri Dec 19 15:51:27 2025 Received: from invmail4.hynix.com (exvmail4.hynix.com [166.125.252.92]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 2E7EA26A0BA; Mon, 19 May 2025 09:18:44 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=166.125.252.92 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747646331; cv=none; b=CpnHSRhqrVztvKHsLx0MZx6SK25B0PexEZVYHZn3mb4dH0wuqEw+F8dqH+clyifm8liB8qDS65B8zt7iVZNzntorzGBwrOZabU+lHJ8VGAuNqJBybLt5NibGSFWy9UdyzHsiBtYmb5FlT6KE9vhrPuPgqwLUzT9gEGzOS1Fm41o= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747646331; c=relaxed/simple; bh=UPgrUdj2zJuIcM2CRwF3Ndg48wh45GAuitrkG1THTag=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References; b=Y2Jd0iqX19oi9SH5Xo4C6mWgqqtxZYQ4UMtjZPTuOsx5lKIqtQd8q+bexlg9xf0XlzfqBuUr2iUDu+h3bzmWPEyBcoIZ1kEHcUsfdvL47Oc3ugVNDuWXap/dwWcqo7L3XyHJUH2cIctdKsKgFE5fls8gdPd20wyMGu6wLfz3CDk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com; spf=pass smtp.mailfrom=sk.com; arc=none smtp.client-ip=166.125.252.92 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=sk.com X-AuditID: a67dfc5b-669ff7000002311f-70-682af76d4e9d From: Byungchul Park To: linux-kernel@vger.kernel.org Cc: kernel_team@skhynix.com, torvalds@linux-foundation.org, damien.lemoal@opensource.wdc.com, linux-ide@vger.kernel.org, adilger.kernel@dilger.ca, linux-ext4@vger.kernel.org, mingo@redhat.com, peterz@infradead.org, will@kernel.org, tglx@linutronix.de, rostedt@goodmis.org, joel@joelfernandes.org, sashal@kernel.org, daniel.vetter@ffwll.ch, duyuyang@gmail.com, johannes.berg@intel.com, tj@kernel.org, tytso@mit.edu, willy@infradead.org, david@fromorbit.com, amir73il@gmail.com, gregkh@linuxfoundation.org, kernel-team@lge.com, linux-mm@kvack.org, akpm@linux-foundation.org, mhocko@kernel.org, minchan@kernel.org, hannes@cmpxchg.org, vdavydov.dev@gmail.com, sj@kernel.org, jglisse@redhat.com, dennis@kernel.org, cl@linux.com, penberg@kernel.org, rientjes@google.com, vbabka@suse.cz, ngupta@vflare.org, linux-block@vger.kernel.org, josef@toxicpanda.com, linux-fsdevel@vger.kernel.org, jack@suse.cz, jlayton@kernel.org, dan.j.williams@intel.com, hch@infradead.org, djwong@kernel.org, dri-devel@lists.freedesktop.org, rodrigosiqueiramelo@gmail.com, melissa.srw@gmail.com, hamohammed.sa@gmail.com, harry.yoo@oracle.com, chris.p.wilson@intel.com, gwan-gyeong.mun@intel.com, max.byungchul.park@gmail.com, boqun.feng@gmail.com, longman@redhat.com, yskelg@gmail.com, yunseong.kim@ericsson.com, yeoreum.yun@arm.com, netdev@vger.kernel.org, matthew.brost@intel.com, her0gyugyu@gmail.com Subject: [PATCH v16 03/42] dept: add single event dependency tracker APIs Date: Mon, 19 May 2025 18:17:47 +0900 Message-Id: <20250519091826.19752-4-byungchul@sk.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20250519091826.19752-1-byungchul@sk.com> References: <20250519091826.19752-1-byungchul@sk.com> X-Brightmail-Tracker: H4sIAAAAAAAAAzWSWUwTYRDH/Xa3u22huKnXKhq0EQ+MKF6ZGM948Pmg8YoPGpVGVtsI1bSK oMGAHHJ7JEAUrLVIbWiR2qLiUYIQLg9EwXIIKKgEwhXRVkE8CpGXyS8z8//NywhJaZVghlCp OsWrVfIQGS2mxH2e+sWqH36KpfkDc8D5PYGC7AIzDbV3TQjMhdEEdJcHQoOrF8GvV69JyEyv RXCrvZWEwoo2BHbjBRrqPntBvXOAhur0ZBpicgpoeNMzQkBLxlUCTNbt8MHQScGLy3oCMrtp yMqMIdyli4AhQx4Dhihf6DBeZ2CkPQCq2xwCsDcvgmvaFhqe2qspqCjqIKDucTYNbea/AnhR UUWBK80baq+kCiC/X09Dj8tAgsE5wMDbEh0BFbqpYIl1C+O//RFAZWoJAfG37xFQ3/QEQXHC RwKsZgcNZc5eAmzWdBKG75Qj6EjrYyAuZYiBrOg0BMlxGRTEtqyEXz/dl298D4DomxYK8n87 0Ia12Kw1I1zWO0DiWNsZPOx8R2O7S0fh53oOP7reyuDY4mYG66ynsc3oh3OedhP41qBTgK15 iTS2Dl5lcFJfPYH7a2qYnTP3i9cE8yHKMF69ZF2QWPHtTQF58iUXrm+zUFHINikJiYQcu4Kr evgQjbOzrokaZZqdzzU2DpGjPJmdzdlSOwVJSCwkWYcH13CjaSwwid3G5TpqxgIU68sN2h/Q oyxhV3JV9y79l/pwJkvJmEjEruKak8vG+lL3Tr1JS41KOTZLxP18a/kfmM49MzZSl5FEhybk IalSFRYqV4as8FdEqJTh/kdOhFqR+78MkSMHitBg7Z5SxAqRzFNisS9USAXyME1EaCnihKRs siTPtkAhlQTLI87y6hOH1adDeE0p8hZSsmmSZa4zwVL2mPwUf5znT/Lq8SkhFM2IQtrdre1I XZj7O6txb9S81+8OafdVWjZvvd8/uzgmZjhyy4NkyZevu1ZvGOn+8inFdLD9qGRj4qvlc6Z5 SX2au/YGpbV6LnJWn4e5kRfBE3/0PR+XQRsmvt+xPmVWz/3yHecKNjV4G70e25ZNSfWwBSVu tvcY/4iGf+BAqkiT8CzbRyqjNAp5gB+p1sj/AV5ct3JbAwAA X-Brightmail-Tracker: H4sIAAAAAAAAAzWSW0xTaRDH/c6das1JJXq8BdPEqBgvGKtDNIT4IJ+a3bjG4OVFGz3axoLY CopmI9iCLFAQIrCiYEGtWIqUFu+UIChaLwiCRQhWIayCoESkrBUUC8aXyS/zn/nNy3CkzELP 4tTRh0VttFIjZySU5M81+iWa/4NVy1uzAbxDKRScL7cy0HitFIG1MpGA3gcR0Drcj2Dk2XMS 8nIaERR1viahst6DwFlykoHm7qnQ4h1gwJWTxoD+YjkDTX2jBHTkZhNQav8D3pjfUfDkdDEB eb0MnMvTE/7SQ4DPbGHBnDAfukryWRjtDAGXx01DXYGLBmf7Yjhb2MFAldNFQf2tLgKa75xn wGMdo+FJ/SMKhjNmQ2OWkYayT8UM9A2bSTB7B1h4UWMioN40HWwGvzX5yw8aHhprCEi+VEFA S9tdBNUpbwmwW90M1Hn7CXDYc0j4duUBgq6MjywkpftYOJeYgSAtKZcCQ4cCRr76LxcMhUDi BRsFZd/dKDwMWwutCNf1D5DY4DiCv3lfMtg5bKLw42IB385/zWJDdTuLTfZY7CgJxhereglc NOilsd3yD4Ptg9ksTv3YQuBPDQ3s5rk7JWv3ihp1nKhdFrZbovrSVE7GPBWOFntsVAJyTEtF AZzArxS8zW3UODP8AuHVKx85zoH8PMFhfEenIglH8u7JQmtBGxoPpvEbhMvuhokFip8vDDpv MOMs5RXCo4pM9EsaJJTaaiZEAfwqoT2tbqIv88+0lBZSp5HEhCZZUKA6Oi5KqdYoluoOqOKj 1UeX7jkYZUf+DzL/PZp1Cw01R9QinkPyKVKbc5FKRivjdPFRtUjgSHmg1OJYqJJJ9yrjj4na g7u0sRpRV4tmc5R8hnTjNnG3jN+vPCweEMUYUfs7JbiAWQlo/fvH2SVhc9Kn/3V8pjFra9Cm 8BlBJ07d69zi6r+Px/IjNpcd8awLtd8c3FHZLSXXNRlWrc46k/6f4qqy73pkQLcvcju3KOlu rr5t/79FxorMrytCZUW96fqx2Ji3oRkjTaNJ+sU+uiL5ekrm2Z6b+w6tGLn6QVHl2Hcq/Pvn 7hs9c6rllE6lDAkmtTrlTyrxfgA9AwAA X-CFilter-Loop: Reflected Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Wrapped the base APIs for easier annotation on wait and event. Start with supporting waiters on each single event. More general support for multiple events is a future work. Do more when the need arises. How to annotate: 1. Initaialize a map for the interesting wait. /* * Place along with the wait instance. */ struct dept_map my_wait; /* * Place in the initialization code. */ sdt_map_init(&my_wait); 2. Place the following at the wait code. sdt_wait(&my_wait); 3. Place the following at the event code. sdt_event(&my_wait); That's it! Signed-off-by: Byungchul Park --- include/linux/dept_sdt.h | 65 ++++++++++++++++++++++++++++++++++++++++ 1 file changed, 65 insertions(+) create mode 100644 include/linux/dept_sdt.h diff --git a/include/linux/dept_sdt.h b/include/linux/dept_sdt.h new file mode 100644 index 000000000000..0535f763b21b --- /dev/null +++ b/include/linux/dept_sdt.h @@ -0,0 +1,65 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* + * Single-event Dependency Tracker + * + * Started by Byungchul Park : + * + * Copyright (c) 2020 LG Electronics, Inc., Byungchul Park + * Copyright (c) 2024 SK hynix, Inc., Byungchul Park + */ + +#ifndef __LINUX_DEPT_SDT_H +#define __LINUX_DEPT_SDT_H + +#include +#include + +#ifdef CONFIG_DEPT +#define sdt_map_init(m) \ + do { \ + static struct dept_key __key; \ + dept_map_init(m, &__key, 0, #m); \ + } while (0) + +#define sdt_map_init_key(m, k) dept_map_init(m, k, 0, #m) + +#define sdt_wait(m) \ + do { \ + dept_request_event(m); \ + dept_wait(m, 1UL, _THIS_IP_, __func__, 0); \ + } while (0) + +/* + * sdt_might_sleep() and its family will be committed in __schedule() + * when it actually gets to __schedule(). Both dept_request_event() and + * dept_wait() will be performed on the commit. + */ + +/* + * Use the code location as the class key if an explicit map is not used. + */ +#define sdt_might_sleep_start(m) \ + do { \ + struct dept_map *__m =3D m; \ + static struct dept_key __key; \ + dept_stage_wait(__m, __m ? NULL : &__key, _THIS_IP_, __func__);\ + } while (0) + +#define sdt_might_sleep_end() dept_clean_stage() + +#define sdt_ecxt_enter(m) dept_ecxt_enter(m, 1UL, _THIS_IP_, "start", "ev= ent", 0) +#define sdt_event(m) dept_event(m, 1UL, _THIS_IP_, __func__) +#define sdt_ecxt_exit(m) dept_ecxt_exit(m, 1UL, _THIS_IP_) +#define sdt_request_event(m) dept_request_event(m) +#else /* !CONFIG_DEPT */ +#define sdt_map_init(m) do { } while (0) +#define sdt_map_init_key(m, k) do { (void)(k); } while (0) +#define sdt_wait(m) do { } while (0) +#define sdt_might_sleep_start(m) do { } while (0) +#define sdt_might_sleep_end() do { } while (0) +#define sdt_ecxt_enter(m) do { } while (0) +#define sdt_event(m) do { } while (0) +#define sdt_ecxt_exit(m) do { } while (0) +#define sdt_request_event(m) do { } while (0) +#endif +#endif /* __LINUX_DEPT_SDT_H */ --=20 2.17.1 From nobody Fri Dec 19 15:51:27 2025 Received: from invmail4.hynix.com (exvmail4.hynix.com [166.125.252.92]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 3693726D4F8; Mon, 19 May 2025 09:18:50 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=166.125.252.92 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747646333; cv=none; b=bai6IUGEs6fV1uQ+rE2Bk/t7Jt3NpyC2SAE/C66Cu7G2Cjje2cgX0g/+I8t+/ndp5NS5zV8nFy53jEP4OXBgUWsQZr30PPcQVUbsXuAJl+Co6iDPF4wZI2KPfL3byPo0s01J72f27AfQwyluGaSn+QPqiG7rkanMdtCfIkyc8gc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747646333; c=relaxed/simple; bh=RX0BVWE27Cost6W7zXdbtvLSq8FC7lANMhyhh7fOZCU=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References; b=iOXj6GhrZq3lCVVkZpVEZecACw/MfBGM0aExdm+Yj1Jy21CEpob9C7bMNDa6+jFUSzacuMQKXBD9OZorPUnbgtmmH+lv7hCVhsHldMgVwIP5j+TF9xFubcfGjF4J5s7JIDKpQXMiEs5/H13tVsG5K2MJ61s3by+Y+vuaAUUvNK4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com; spf=pass smtp.mailfrom=sk.com; arc=none smtp.client-ip=166.125.252.92 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=sk.com X-AuditID: a67dfc5b-681ff7000002311f-88-682af76e3e16 From: Byungchul Park To: linux-kernel@vger.kernel.org Cc: kernel_team@skhynix.com, torvalds@linux-foundation.org, damien.lemoal@opensource.wdc.com, linux-ide@vger.kernel.org, adilger.kernel@dilger.ca, linux-ext4@vger.kernel.org, mingo@redhat.com, peterz@infradead.org, will@kernel.org, tglx@linutronix.de, rostedt@goodmis.org, joel@joelfernandes.org, sashal@kernel.org, daniel.vetter@ffwll.ch, duyuyang@gmail.com, johannes.berg@intel.com, tj@kernel.org, tytso@mit.edu, willy@infradead.org, david@fromorbit.com, amir73il@gmail.com, gregkh@linuxfoundation.org, kernel-team@lge.com, linux-mm@kvack.org, akpm@linux-foundation.org, mhocko@kernel.org, minchan@kernel.org, hannes@cmpxchg.org, vdavydov.dev@gmail.com, sj@kernel.org, jglisse@redhat.com, dennis@kernel.org, cl@linux.com, penberg@kernel.org, rientjes@google.com, vbabka@suse.cz, ngupta@vflare.org, linux-block@vger.kernel.org, josef@toxicpanda.com, linux-fsdevel@vger.kernel.org, jack@suse.cz, jlayton@kernel.org, dan.j.williams@intel.com, hch@infradead.org, djwong@kernel.org, dri-devel@lists.freedesktop.org, rodrigosiqueiramelo@gmail.com, melissa.srw@gmail.com, hamohammed.sa@gmail.com, harry.yoo@oracle.com, chris.p.wilson@intel.com, gwan-gyeong.mun@intel.com, max.byungchul.park@gmail.com, boqun.feng@gmail.com, longman@redhat.com, yskelg@gmail.com, yunseong.kim@ericsson.com, yeoreum.yun@arm.com, netdev@vger.kernel.org, matthew.brost@intel.com, her0gyugyu@gmail.com Subject: [PATCH v16 04/42] dept: add lock dependency tracker APIs Date: Mon, 19 May 2025 18:17:48 +0900 Message-Id: <20250519091826.19752-5-byungchul@sk.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20250519091826.19752-1-byungchul@sk.com> References: <20250519091826.19752-1-byungchul@sk.com> X-Brightmail-Tracker: H4sIAAAAAAAAAzWSa0yTZxTHed5rW6150+n2KCYzLV5gGYqB5czLNuMHn0RNTPxghKh0452t lkpaBTExgoIXGAQxpU6lFnBdpVVYq4tOS5DOIopasOM2YIMsKEJlAVqt4raC88vJL/n/z+98 ORJa4WMXSLT6/aJBr9YpORkjC86u+lT/MkGzom1IBaHJkwxcqHNy4L/qQOC8lk/B8N0N0Bke RfDm4WMazCY/gqqBPhqu+foReOxHOXjy1xwIhMY4aDEVc3Cspo6DtpEpCnoryilwuDbDH7Yh Bh6UVVNgHubgvPkYFR3PKIjYanmw5S2GQfs5HqYGkqClv4MFT88n8L2ll4PbnhYGfDcGKXjy ywUO+p3/svDAd4+BcGks+E+XsHDlRTUHI2EbDbbQGA/tjVYKfNYPob4gKjw+8Q8LzSWNFBy/ 9BMFge5bCBpO/kmBy9nBgTc0SoHbZaLh9Y93EQyWBnko/C7Cw/n8UgTFhRUMFPSmwJtX0cuV k0mQf7GegStvO9BXa4nT4kTEOzpGkwJ3Dnkd+o0jnrCVIferMbl5ro8nBQ09PLG6DhC3PYHU 3B6mSNV4iCWu2lMccY2X86QoGKDIi0eP+C0LU2VrMkSdNls0LP8iXaYZaf86qxIfzH/uYPNQ QFGEpBIsJOOxy2bqPQf9rfw0c8JS3NUVoad5rrAIu0uG2CIkk9BCxyzcWdmNpoMPhHW486iZ mWZGWIy99t9nRHIhBf/wrP1/6cfYUd84I5IKn+GeYu/MriLaCTgszLvOGSm2VK54x/PxHXsX U4bkVhRTixRafXamWqtLTtTk6rUHE7/Zl+lC0e+yHZ5Ku4HG/VubkCBBytnyek+8RsGqs425 mU0IS2jlXHmte5lGIc9Q5x4SDft2GQ7oRGMTipUwyo/kK8M5GQpht3q/uFcUs0TD+5SSSBfk ofIlqp3xa+Jb519ETyOW3k2kbdvKhWd1934tnDWQCV6T1pzimuxLmShTSb9NG8nxT1xPL1m9 8VZy1qEKHFNcuD5uouXt36rRvsgcZeud3TdXmbZtjTvVfGL9KmlD7HheSLVhT6lwZvthbNR2 ueNqtl+P2eGomtf9+eMjR4I/r/4y1a5kjBp1UgJtMKr/AwUD32NZAwAA X-Brightmail-Tracker: H4sIAAAAAAAAAzWSe0hTcRTH+/3ua65WlyV5e1EMpDIyDY0DRRRE3swiIggiqJWXNpzTtjKN Ik17qZMUVCpny2ytbdXapKczc2Qt8zWXlpmVlGm+Ip1p2mMr+ufwge/3fM4/R0RIr1NzREr1 QUGjlqtktJgUb1mVuSzxe5gioiwfg2/kDAklt6w0NN20ILBWZGDofRIDbaP9CCbqGwkoLmxC cPnDWwIqajsROE0naGj5OB28viEa3IU5NGReuUVDc98kho6iAgwW+2Z4Z+wmoe5cGYbiXhou Fmdi/+jBMG40M2BMD4Uu0wUGJj9EgruzlQKX3k2Bs30pnC/toKHS6Sah9l4XhpYHJTR0Wn9T UFf7jITRvLnQlK+j4MZgGQ19o0YCjL4hBjzVBgy1hllgy/JbTw3/ouCprhrDqfLbGLyvHyKo OvMeg93aSoPL14/BYS8k4Me1Jwi68gYYOJk7zsDFjDwEOSeLSMjqiIaJMf9l/UgkZFyykXDj Zytau4a3lloR7+ofIvgsx2H+h+8lzTtHDST/vIzj7194y/BZVe0Mb7Af4h2mMP5KZS/mL3/z UbzdfJbm7d8KGD57wIv5wYYGZuv8neLV8YJKmSJolq/ZI1b0efYm67nUjC8WKh15pdkoSMSx UdxA0wsmwDS7iHv1apwIcDC7kHPouqlsJBYRbOtUrk3/GgWCmew6ru1EMRlgkg3lXKY3OMAS Npq72uPB/6QLOIut+q8oiF3Jtee4/u5K/R2vpZQ8h8QGNMWMgpXqlES5UhUdrk1QpKmVqeH7 khLtyP9AxmOT+ffQSEtMDWJFSDZNYnMuUUgpeYo2LbEGcSJCFiwxOxYrpJJ4edoRQZO0W3NI JWhr0FwRKQuRxO4Q9kjZ/fKDQoIgJAua/ykWBc1JRxHl5XW76u2f5sXE6j1JURvaqgy6RQO2 XG/97pDGr4/jmmXO4/qPLmPJbJWvIUSy/deO3m7m5wpZ1Ontm47mP+gJNc1vbrwdPLYx7o5y cFvyi62PwjYU3V844+GENuLZtOHYsbu6hJDKgrgD6vXYszjXvNnxeTjcSEe7E85zqTGbkmSk ViGPDCM0WvkfHkKg8jwDAAA= X-CFilter-Loop: Reflected Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Wrap the base APIs for easier annotation on typical lock. Signed-off-by: Byungchul Park --- include/linux/dept_ldt.h | 78 ++++++++++++++++++++++++++++++++++++++++ 1 file changed, 78 insertions(+) create mode 100644 include/linux/dept_ldt.h diff --git a/include/linux/dept_ldt.h b/include/linux/dept_ldt.h new file mode 100644 index 000000000000..8047d0a531f1 --- /dev/null +++ b/include/linux/dept_ldt.h @@ -0,0 +1,78 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* + * Lock Dependency Tracker + * + * Started by Byungchul Park : + * + * Copyright (c) 2020 LG Electronics, Inc., Byungchul Park + * Copyright (c) 2024 SK hynix, Inc., Byungchul Park + */ + +#ifndef __LINUX_DEPT_LDT_H +#define __LINUX_DEPT_LDT_H + +#include + +#ifdef CONFIG_DEPT +#define LDT_EVT_L 1UL +#define LDT_EVT_R 2UL +#define LDT_EVT_W 1UL +#define LDT_EVT_RW (LDT_EVT_R | LDT_EVT_W) +#define LDT_EVT_ALL (LDT_EVT_L | LDT_EVT_RW) + +#define ldt_init(m, k, su, n) dept_map_init(m, k, su, n) +#define ldt_lock(m, sl, t, n, i) \ + do { \ + if (n) \ + dept_ecxt_enter_nokeep(m); \ + else if (t) \ + dept_ecxt_enter(m, LDT_EVT_L, i, "trylock", "unlock", sl);\ + else { \ + dept_wait(m, LDT_EVT_L, i, "lock", sl); \ + dept_ecxt_enter(m, LDT_EVT_L, i, "lock", "unlock", sl);\ + } \ + } while (0) + +#define ldt_rlock(m, sl, t, n, i, q) \ + do { \ + if (n) \ + dept_ecxt_enter_nokeep(m); \ + else if (t) \ + dept_ecxt_enter(m, LDT_EVT_R, i, "read_trylock", "read_unlock", sl);\ + else { \ + dept_wait(m, q ? LDT_EVT_RW : LDT_EVT_W, i, "read_lock", sl);\ + dept_ecxt_enter(m, LDT_EVT_R, i, "read_lock", "read_unlock", sl);\ + } \ + } while (0) + +#define ldt_wlock(m, sl, t, n, i) \ + do { \ + if (n) \ + dept_ecxt_enter_nokeep(m); \ + else if (t) \ + dept_ecxt_enter(m, LDT_EVT_W, i, "write_trylock", "write_unlock", sl);\ + else { \ + dept_wait(m, LDT_EVT_RW, i, "write_lock", sl); \ + dept_ecxt_enter(m, LDT_EVT_W, i, "write_lock", "write_unlock", sl);\ + } \ + } while (0) + +#define ldt_unlock(m, i) dept_ecxt_exit(m, LDT_EVT_ALL, i) + +#define ldt_downgrade(m, i) \ + do { \ + if (dept_ecxt_holding(m, LDT_EVT_W)) \ + dept_map_ecxt_modify(m, LDT_EVT_W, NULL, LDT_EVT_R, i, "downgrade", "re= ad_unlock", -1);\ + } while (0) + +#define ldt_set_class(m, n, k, sl, i) dept_map_ecxt_modify(m, LDT_EVT_ALL,= k, 0UL, i, "lock_set_class", "(any)unlock", sl) +#else /* !CONFIG_DEPT */ +#define ldt_init(m, k, su, n) do { (void)(k); } while (0) +#define ldt_lock(m, sl, t, n, i) do { } while (0) +#define ldt_rlock(m, sl, t, n, i, q) do { } while (0) +#define ldt_wlock(m, sl, t, n, i) do { } while (0) +#define ldt_unlock(m, i) do { } while (0) +#define ldt_downgrade(m, i) do { } while (0) +#define ldt_set_class(m, n, k, sl, i) do { } while (0) +#endif +#endif /* __LINUX_DEPT_LDT_H */ --=20 2.17.1 From nobody Fri Dec 19 15:51:27 2025 Received: from invmail4.hynix.com (exvmail4.hynix.com [166.125.252.92]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 4D45226E14C; Mon, 19 May 2025 09:18:50 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=166.125.252.92 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747646334; cv=none; b=dWunZ+vr8Y4zyKR6VR/kZUhS5UGbNUOrYAxE2+z2Anw4AUM8lOaeRtTc//eULu4IvpyEMFM2i+6xfWBmw/FtJS3ilZER9vCFLsbUnXwqKDfn5K5ejkWz7lOkVzqZiJdsS0CdWx8eLjose06LMzrGryx+Lr8gJCPTRGvw7whBYko= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747646334; c=relaxed/simple; bh=sdBf+jdap8ZYMWI+dmhqrnpwWrwLomAGjaFbuF2ajZI=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References; b=ABSznEzO1SM8qGLKl/mQexHfPZ6OyxSBZ3Ez0E5IkgceFiXdqU3sp3KKxm6kuQ5gBS9x4dzcvhGkVvat5AfU2+pE/2h0hhRX/fjvbAMZupIixkz0zNF2zRj1WVYz9VNfE2qwnfpKJXBNNbj8KuRFkiOHktThJ2dVuXjwgnpHl04= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com; spf=pass smtp.mailfrom=sk.com; arc=none smtp.client-ip=166.125.252.92 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=sk.com X-AuditID: a67dfc5b-669ff7000002311f-83-682af76e49b0 From: Byungchul Park To: linux-kernel@vger.kernel.org Cc: kernel_team@skhynix.com, torvalds@linux-foundation.org, damien.lemoal@opensource.wdc.com, linux-ide@vger.kernel.org, adilger.kernel@dilger.ca, linux-ext4@vger.kernel.org, mingo@redhat.com, peterz@infradead.org, will@kernel.org, tglx@linutronix.de, rostedt@goodmis.org, joel@joelfernandes.org, sashal@kernel.org, daniel.vetter@ffwll.ch, duyuyang@gmail.com, johannes.berg@intel.com, tj@kernel.org, tytso@mit.edu, willy@infradead.org, david@fromorbit.com, amir73il@gmail.com, gregkh@linuxfoundation.org, kernel-team@lge.com, linux-mm@kvack.org, akpm@linux-foundation.org, mhocko@kernel.org, minchan@kernel.org, hannes@cmpxchg.org, vdavydov.dev@gmail.com, sj@kernel.org, jglisse@redhat.com, dennis@kernel.org, cl@linux.com, penberg@kernel.org, rientjes@google.com, vbabka@suse.cz, ngupta@vflare.org, linux-block@vger.kernel.org, josef@toxicpanda.com, linux-fsdevel@vger.kernel.org, jack@suse.cz, jlayton@kernel.org, dan.j.williams@intel.com, hch@infradead.org, djwong@kernel.org, dri-devel@lists.freedesktop.org, rodrigosiqueiramelo@gmail.com, melissa.srw@gmail.com, hamohammed.sa@gmail.com, harry.yoo@oracle.com, chris.p.wilson@intel.com, gwan-gyeong.mun@intel.com, max.byungchul.park@gmail.com, boqun.feng@gmail.com, longman@redhat.com, yskelg@gmail.com, yunseong.kim@ericsson.com, yeoreum.yun@arm.com, netdev@vger.kernel.org, matthew.brost@intel.com, her0gyugyu@gmail.com Subject: [PATCH v16 05/42] dept: tie to lockdep and IRQ tracing Date: Mon, 19 May 2025 18:17:49 +0900 Message-Id: <20250519091826.19752-6-byungchul@sk.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20250519091826.19752-1-byungchul@sk.com> References: <20250519091826.19752-1-byungchul@sk.com> X-Brightmail-Tracker: H4sIAAAAAAAAAzWSbUxTZxTH99yX516KJTedzjs121LndBid+DKPy3B80ueLxsSYvZhF67ja Cq1aFMRJArQQlZX4MiRDxFK0w7ZIbdnGnKUVAg7IpEJFRERkhthYQCrtRNhmwfjl5Jec//93 vhyeVtxk5/Ea3UFJr1OlK7GMkQ3Pqlym+ydRveJGw0KIjB9joLzWgcF/xY7AUZdHQbB5I9yN hhBM/tVBQ2mJH0Hlowc01LX0I/BU52PoepwAgcgohtaSIgyGqloMt59OUdB39jQFdtcmeGgd YqD9pIWC0iCGc6UGKjaeUDBhtXFgzV0Eg9VlHEw9SoLW/m4WPL1L4aeKPgzXPa0MtNQPUtB1 rRxDv+N/Ftpb/mQgWjwf/KdMLNSMWDA8jVppsEZGOej0mSloMb8DTmNMWPj8PxZumnwUFF68 SkHg3h8IGo4NUOBydGNoioQocLtKaHj5czOCweJhDgp+mODgXF4xgqKCswwY+9bA5IvY5fPj SZB3wclAzb/dKCWZOCociDSFRmlidGeRl5E7mHiiZoa0WUTye9kDjhgbejlidh0i7upEUnU9 SJHKcIQlLttxTFzh0xw5MRygyMitW9yWBd/IPk+V0jWZkv6T9Ttl6tpTRry/8/vDzoEFuejy rhMojheF1eLItTr2DV8uN6FpxsJisadngp7m2cIHots0FMvIeFrojhfvnr83E3pb+EL8xRea KTPCIjE4NoCnWS6sEZvywsxr6fui3embEcUJn4q9RU0zXUUsE7BXMNNSUTgTJ0YNXu514V3x RnUPcxLJzegtG1JodJlalSZ99XJ1tk5zePl3+7QuFHsva87U9noU9m9tRAKPlLPkTs/HagWr yszI1jYikaeVs+U29xK1Qp6qyj4i6fft0B9KlzIa0XyeUc6Vr4xmpSqEPaqDUpok7Zf0b7YU HzcvF6XYs75aN75Zd/UzQ4cvfn3pAe2QEP9b38qa/HxTecrRXduLatstL+w5hT92blj1XoK0 t8qwra3Xz2q9W8b+TmPjlxVc+vBxx5fYIk8IpfWT8YFiPlDYkRxuG6v35nu/Xrvx6P07XlvK cFd455xvV3yU0zyZfGTI++ue3RWbnpWNBZ8omQy1KimR1meoXgE99uKMWgMAAA== X-Brightmail-Tracker: H4sIAAAAAAAAAzWSa0iTYRTHe97rXC3eptVrF4pBVJMsIeNAZVFED1IRBEZ9yZUvbuXMtjJX BJpTSpu0wOyitlSWbrOtrYu3iTlaLakszUpsmUmlriRzlmbWNPpy+MH5n9/58heR0ip6nkiV elTQpCpSZIyYEu9Ym71C/UOuXNU7IIbg8BkKiu02BlpvWhHYbmcR0PdgK7waCSD49eQZCUWF rQiuv39Lwm2vH4G78jQDbb0zoT04yICvMJ+B7HI7A88HxgnouniBAKtzO7wzf6Sg5XwZAUV9 DFwtyiZC4zMBo2YLC+bMJdBTeYWF8fcx4PN30OAp8dHg7oyCy6VdDDS4fRR4a3oIaKsrZsBv +0NDi/cRBSMF86HVaKCh+msZAwMjZhLMwUEWXjSZCPCa5oBDH7Lmfp+g4aGhiYDcilsEtL+p R9B4ppsAp62DAU8wQIDLWUjC2I0HCHoKvrCQc26UhatZBQjycy5SoO+KhV8/Q59LhmMg65qD gurfHWhjHLaV2hD2BAZJrHcdx2PBlwx2j5go/LiMx7VX3rJY39jJYpPzGHZVynF5Qx+Brw8F aey0nGWwc+gCi/O+tBP469On7M6Fe8XrkoQUVbqgWRmXKFbajXom7cXJDEf3gkxUtT8PhYl4 bjVfVWxAk8xwS/nXr0fJSY7gFvMuw0c6D4lFJNcxnX9V8mYqFM5t4O80BehJprglfN+3bmaS JVws78kaov5JF/FWR9OUKIxbw3fme6ZupaFMu7WUOo/EJjTNgiJUqelqhSolNlp7SKlLVWVE HzisdqJQgcynxo01aLhtazPiREg2Q+JwL1dKaUW6VqduRryIlEVILK5lSqkkSaE7IWgO79Mc SxG0zWi+iJLNlcTvFhKlXLLiqHBIENIEzf8tIQqbl4n29XPv/KVr1yfMvlRhiKzafOSg0fwn 7sa5wKyYnLsvySj3Hfp+MEmuG0vwfVh0urp1qTpuD7Etvj9iS1StxRupS65ITOv+NLO+ruWg wmwb6/c1sJvisTGHlifU197rtO6PDPc/nnimXP2odyAsY/nvaPs0tc4+kZzbz+4yqQuKZJRW qYiRkxqt4i9Scb27PAMAAA== X-CFilter-Loop: Reflected Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" How to place dept this way looks so ugly. However, it's inevitable for now. The way should be enhanced gradually. Signed-off-by: Byungchul Park --- include/linux/irqflags.h | 7 +- include/linux/local_lock_internal.h | 1 + include/linux/lockdep.h | 102 ++++++++++++++++++++++------ include/linux/lockdep_types.h | 3 + include/linux/mutex.h | 1 + include/linux/percpu-rwsem.h | 2 +- include/linux/rtmutex.h | 1 + include/linux/rwlock_types.h | 1 + include/linux/rwsem.h | 1 + include/linux/seqlock.h | 2 +- include/linux/spinlock_types_raw.h | 3 + include/linux/srcu.h | 2 +- kernel/dependency/dept.c | 8 +-- kernel/locking/lockdep.c | 22 ++++++ 14 files changed, 127 insertions(+), 29 deletions(-) diff --git a/include/linux/irqflags.h b/include/linux/irqflags.h index 57b074e0cfbb..d8b9cf093f83 100644 --- a/include/linux/irqflags.h +++ b/include/linux/irqflags.h @@ -15,6 +15,7 @@ #include #include #include +#include #include #include =20 @@ -55,8 +56,10 @@ extern void trace_hardirqs_off(void); # define lockdep_softirqs_enabled(p) ((p)->softirqs_enabled) # define lockdep_hardirq_enter() \ do { \ - if (__this_cpu_inc_return(hardirq_context) =3D=3D 1)\ + if (__this_cpu_inc_return(hardirq_context) =3D=3D 1) { \ current->hardirq_threaded =3D 0; \ + dept_hardirq_enter(); \ + } \ } while (0) # define lockdep_hardirq_threaded() \ do { \ @@ -131,6 +134,8 @@ do { \ # define lockdep_softirq_enter() \ do { \ current->softirq_context++; \ + if (current->softirq_context =3D=3D 1) \ + dept_softirq_enter(); \ } while (0) # define lockdep_softirq_exit() \ do { \ diff --git a/include/linux/local_lock_internal.h b/include/linux/local_lock= _internal.h index 8d5ac16a9b17..ec3ff5931aa6 100644 --- a/include/linux/local_lock_internal.h +++ b/include/linux/local_lock_internal.h @@ -27,6 +27,7 @@ typedef struct { .name =3D #lockname, \ .wait_type_inner =3D LD_WAIT_CONFIG, \ .lock_type =3D LD_LOCK_PERCPU, \ + .dmap =3D DEPT_MAP_INITIALIZER(lockname, NULL),\ }, \ .owner =3D NULL, =20 diff --git a/include/linux/lockdep.h b/include/linux/lockdep.h index 67964dc4db95..ef03d8808c10 100644 --- a/include/linux/lockdep.h +++ b/include/linux/lockdep.h @@ -12,6 +12,7 @@ =20 #include #include +#include #include =20 struct task_struct; @@ -39,6 +40,8 @@ static inline void lockdep_copy_map(struct lockdep_map *t= o, */ for (i =3D 0; i < NR_LOCKDEP_CACHING_CLASSES; i++) to->class_cache[i] =3D NULL; + + dept_map_copy(&to->dmap, &from->dmap); } =20 /* @@ -428,7 +431,8 @@ enum xhlock_context_t { * Note that _name must not be NULL. */ #define STATIC_LOCKDEP_MAP_INIT(_name, _key) \ - { .name =3D (_name), .key =3D (void *)(_key), } + { .name =3D (_name), .key =3D (void *)(_key), \ + .dmap =3D DEPT_MAP_INITIALIZER(_name, _key) } =20 static inline void lockdep_invariant_state(bool force) {} static inline void lockdep_free_task(struct task_struct *task) {} @@ -510,33 +514,89 @@ extern bool read_lock_is_recursive(void); #define lock_acquire_shared(l, s, t, n, i) lock_acquire(l, s, t, 1, 1, n,= i) #define lock_acquire_shared_recursive(l, s, t, n, i) lock_acquire(l, s, t,= 2, 1, n, i) =20 -#define spin_acquire(l, s, t, i) lock_acquire_exclusive(l, s, t, NULL, i) -#define spin_acquire_nest(l, s, t, n, i) lock_acquire_exclusive(l, s, t, n= , i) -#define spin_release(l, i) lock_release(l, i) - -#define rwlock_acquire(l, s, t, i) lock_acquire_exclusive(l, s, t, NULL, = i) +#define spin_acquire(l, s, t, i) \ +do { \ + ldt_lock(&(l)->dmap, s, t, NULL, i); \ + lock_acquire_exclusive(l, s, t, NULL, i); \ +} while (0) +#define spin_acquire_nest(l, s, t, n, i) \ +do { \ + ldt_lock(&(l)->dmap, s, t, n, i); \ + lock_acquire_exclusive(l, s, t, n, i); \ +} while (0) +#define spin_release(l, i) \ +do { \ + ldt_unlock(&(l)->dmap, i); \ + lock_release(l, i); \ +} while (0) +#define rwlock_acquire(l, s, t, i) \ +do { \ + ldt_wlock(&(l)->dmap, s, t, NULL, i); \ + lock_acquire_exclusive(l, s, t, NULL, i); \ +} while (0) #define rwlock_acquire_read(l, s, t, i) \ do { \ + ldt_rlock(&(l)->dmap, s, t, NULL, i, !read_lock_is_recursive());\ if (read_lock_is_recursive()) \ lock_acquire_shared_recursive(l, s, t, NULL, i); \ else \ lock_acquire_shared(l, s, t, NULL, i); \ } while (0) - -#define rwlock_release(l, i) lock_release(l, i) - -#define seqcount_acquire(l, s, t, i) lock_acquire_exclusive(l, s, t, NULL= , i) -#define seqcount_acquire_read(l, s, t, i) lock_acquire_shared_recursive(l,= s, t, NULL, i) -#define seqcount_release(l, i) lock_release(l, i) - -#define mutex_acquire(l, s, t, i) lock_acquire_exclusive(l, s, t, NULL, i) -#define mutex_acquire_nest(l, s, t, n, i) lock_acquire_exclusive(l, s, t, = n, i) -#define mutex_release(l, i) lock_release(l, i) - -#define rwsem_acquire(l, s, t, i) lock_acquire_exclusive(l, s, t, NULL, i) -#define rwsem_acquire_nest(l, s, t, n, i) lock_acquire_exclusive(l, s, t, = n, i) -#define rwsem_acquire_read(l, s, t, i) lock_acquire_shared(l, s, t, NULL,= i) -#define rwsem_release(l, i) lock_release(l, i) +#define rwlock_release(l, i) \ +do { \ + ldt_unlock(&(l)->dmap, i); \ + lock_release(l, i); \ +} while (0) +#define seqcount_acquire(l, s, t, i) \ +do { \ + ldt_wlock(&(l)->dmap, s, t, NULL, i); \ + lock_acquire_exclusive(l, s, t, NULL, i); \ +} while (0) +#define seqcount_acquire_read(l, s, t, i) \ +do { \ + ldt_rlock(&(l)->dmap, s, t, NULL, i, false); \ + lock_acquire_shared_recursive(l, s, t, NULL, i); \ +} while (0) +#define seqcount_release(l, i) \ +do { \ + ldt_unlock(&(l)->dmap, i); \ + lock_release(l, i); \ +} while (0) +#define mutex_acquire(l, s, t, i) \ +do { \ + ldt_lock(&(l)->dmap, s, t, NULL, i); \ + lock_acquire_exclusive(l, s, t, NULL, i); \ +} while (0) +#define mutex_acquire_nest(l, s, t, n, i) \ +do { \ + ldt_lock(&(l)->dmap, s, t, n, i); \ + lock_acquire_exclusive(l, s, t, n, i); \ +} while (0) +#define mutex_release(l, i) \ +do { \ + ldt_unlock(&(l)->dmap, i); \ + lock_release(l, i); \ +} while (0) +#define rwsem_acquire(l, s, t, i) \ +do { \ + ldt_lock(&(l)->dmap, s, t, NULL, i); \ + lock_acquire_exclusive(l, s, t, NULL, i); \ +} while (0) +#define rwsem_acquire_nest(l, s, t, n, i) \ +do { \ + ldt_lock(&(l)->dmap, s, t, n, i); \ + lock_acquire_exclusive(l, s, t, n, i); \ +} while (0) +#define rwsem_acquire_read(l, s, t, i) \ +do { \ + ldt_lock(&(l)->dmap, s, t, NULL, i); \ + lock_acquire_shared(l, s, t, NULL, i); \ +} while (0) +#define rwsem_release(l, i) \ +do { \ + ldt_unlock(&(l)->dmap, i); \ + lock_release(l, i); \ +} while (0) =20 #define lock_map_acquire(l) lock_acquire_exclusive(l, 0, 0, NULL, _THIS_= IP_) #define lock_map_acquire_try(l) lock_acquire_exclusive(l, 0, 1, NULL, _T= HIS_IP_) diff --git a/include/linux/lockdep_types.h b/include/linux/lockdep_types.h index 9f361d3ab9d9..6ef4dc67efb3 100644 --- a/include/linux/lockdep_types.h +++ b/include/linux/lockdep_types.h @@ -11,6 +11,7 @@ #define __LINUX_LOCKDEP_TYPES_H =20 #include +#include =20 #define MAX_LOCKDEP_SUBCLASSES 8UL =20 @@ -77,6 +78,7 @@ struct lock_class_key { struct hlist_node hash_entry; struct lockdep_subclass_key subkeys[MAX_LOCKDEP_SUBCLASSES]; }; + struct dept_key dkey; }; =20 extern struct lock_class_key __lockdep_no_validate__; @@ -195,6 +197,7 @@ struct lockdep_map { int cpu; unsigned long ip; #endif + struct dept_map dmap; }; =20 struct pin_cookie { unsigned int val; }; diff --git a/include/linux/mutex.h b/include/linux/mutex.h index 2143d05116be..f3ae3b11e7af 100644 --- a/include/linux/mutex.h +++ b/include/linux/mutex.h @@ -29,6 +29,7 @@ struct device; , .dep_map =3D { \ .name =3D #lockname, \ .wait_type_inner =3D LD_WAIT_SLEEP, \ + .dmap =3D DEPT_MAP_INITIALIZER(lockname, NULL),\ } #else # define __DEP_MAP_MUTEX_INITIALIZER(lockname) diff --git a/include/linux/percpu-rwsem.h b/include/linux/percpu-rwsem.h index af7d75ede619..857b0d46f6f1 100644 --- a/include/linux/percpu-rwsem.h +++ b/include/linux/percpu-rwsem.h @@ -22,7 +22,7 @@ struct percpu_rw_semaphore { }; =20 #ifdef CONFIG_DEBUG_LOCK_ALLOC -#define __PERCPU_RWSEM_DEP_MAP_INIT(lockname) .dep_map =3D { .name =3D #lo= ckname }, +#define __PERCPU_RWSEM_DEP_MAP_INIT(lockname) .dep_map =3D { .name =3D #lo= ckname, .dmap =3D DEPT_MAP_INITIALIZER(lockname, NULL) }, #else #define __PERCPU_RWSEM_DEP_MAP_INIT(lockname) #endif diff --git a/include/linux/rtmutex.h b/include/linux/rtmutex.h index 7d049883a08a..35889ac5eeae 100644 --- a/include/linux/rtmutex.h +++ b/include/linux/rtmutex.h @@ -81,6 +81,7 @@ do { \ .dep_map =3D { \ .name =3D #mutexname, \ .wait_type_inner =3D LD_WAIT_SLEEP, \ + .dmap =3D DEPT_MAP_INITIALIZER(mutexname, NULL),\ } #else #define __DEP_MAP_RT_MUTEX_INITIALIZER(mutexname) diff --git a/include/linux/rwlock_types.h b/include/linux/rwlock_types.h index 1948442e7750..6e58dfc84997 100644 --- a/include/linux/rwlock_types.h +++ b/include/linux/rwlock_types.h @@ -10,6 +10,7 @@ .dep_map =3D { \ .name =3D #lockname, \ .wait_type_inner =3D LD_WAIT_CONFIG, \ + .dmap =3D DEPT_MAP_INITIALIZER(lockname, NULL), \ } #else # define RW_DEP_MAP_INIT(lockname) diff --git a/include/linux/rwsem.h b/include/linux/rwsem.h index c8b543d428b0..2540b18e3489 100644 --- a/include/linux/rwsem.h +++ b/include/linux/rwsem.h @@ -22,6 +22,7 @@ .dep_map =3D { \ .name =3D #lockname, \ .wait_type_inner =3D LD_WAIT_SLEEP, \ + .dmap =3D DEPT_MAP_INITIALIZER(lockname, NULL),\ }, #else # define __RWSEM_DEP_MAP_INIT(lockname) diff --git a/include/linux/seqlock.h b/include/linux/seqlock.h index 5ce48eab7a2a..5f3447449fe0 100644 --- a/include/linux/seqlock.h +++ b/include/linux/seqlock.h @@ -51,7 +51,7 @@ static inline void __seqcount_init(seqcount_t *s, const c= har *name, #ifdef CONFIG_DEBUG_LOCK_ALLOC =20 # define SEQCOUNT_DEP_MAP_INIT(lockname) \ - .dep_map =3D { .name =3D #lockname } + .dep_map =3D { .name =3D #lockname, .dmap =3D DEPT_MAP_INITIALIZER(lockn= ame, NULL) } =20 /** * seqcount_init() - runtime initializer for seqcount_t diff --git a/include/linux/spinlock_types_raw.h b/include/linux/spinlock_ty= pes_raw.h index 91cb36b65a17..3dcc551ded25 100644 --- a/include/linux/spinlock_types_raw.h +++ b/include/linux/spinlock_types_raw.h @@ -31,11 +31,13 @@ typedef struct raw_spinlock { .dep_map =3D { \ .name =3D #lockname, \ .wait_type_inner =3D LD_WAIT_SPIN, \ + .dmap =3D DEPT_MAP_INITIALIZER(lockname, NULL),\ } # define SPIN_DEP_MAP_INIT(lockname) \ .dep_map =3D { \ .name =3D #lockname, \ .wait_type_inner =3D LD_WAIT_CONFIG, \ + .dmap =3D DEPT_MAP_INITIALIZER(lockname, NULL),\ } =20 # define LOCAL_SPIN_DEP_MAP_INIT(lockname) \ @@ -43,6 +45,7 @@ typedef struct raw_spinlock { .name =3D #lockname, \ .wait_type_inner =3D LD_WAIT_CONFIG, \ .lock_type =3D LD_LOCK_PERCPU, \ + .dmap =3D DEPT_MAP_INITIALIZER(lockname, NULL),\ } #else # define RAW_SPIN_DEP_MAP_INIT(lockname) diff --git a/include/linux/srcu.h b/include/linux/srcu.h index 900b0d5c05f5..f2903bdc8179 100644 --- a/include/linux/srcu.h +++ b/include/linux/srcu.h @@ -35,7 +35,7 @@ int __init_srcu_struct(struct srcu_struct *ssp, const cha= r *name, __init_srcu_struct((ssp), #ssp, &__srcu_key); \ }) =20 -#define __SRCU_DEP_MAP_INIT(srcu_name) .dep_map =3D { .name =3D #srcu_name= }, +#define __SRCU_DEP_MAP_INIT(srcu_name) .dep_map =3D { .name =3D #srcu_name= , .dmap =3D DEPT_MAP_INITIALIZER(srcu_name, NULL) }, #else /* #ifdef CONFIG_DEBUG_LOCK_ALLOC */ =20 int init_srcu_struct(struct srcu_struct *ssp); diff --git a/kernel/dependency/dept.c b/kernel/dependency/dept.c index a6671b13891f..6cdda00411bc 100644 --- a/kernel/dependency/dept.c +++ b/kernel/dependency/dept.c @@ -249,10 +249,10 @@ static bool dept_working(void) * Even k =3D=3D NULL is considered as a valid key because it would use * &->map_key as the key in that case. */ -struct dept_key __dept_no_validate__; +extern struct lock_class_key __lockdep_no_validate__; static bool valid_key(struct dept_key *k) { - return &__dept_no_validate__ !=3D k; + return &__lockdep_no_validate__.dkey !=3D k; } =20 /* @@ -1946,7 +1946,7 @@ void dept_softirqs_off(void) dept_task()->softirqs_enabled =3D false; } =20 -void dept_hardirqs_off(void) +void noinstr dept_hardirqs_off(void) { /* * Assumes that it's called with IRQ disabled so that accessing @@ -1968,7 +1968,7 @@ void dept_softirq_enter(void) /* * Ensure it's the outmost hardirq context. */ -void dept_hardirq_enter(void) +void noinstr dept_hardirq_enter(void) { struct dept_task *dt =3D dept_task(); =20 diff --git a/kernel/locking/lockdep.c b/kernel/locking/lockdep.c index 58d78a33ac65..6c984a55d5ed 100644 --- a/kernel/locking/lockdep.c +++ b/kernel/locking/lockdep.c @@ -1226,6 +1226,8 @@ void lockdep_register_key(struct lock_class_key *key) struct lock_class_key *k; unsigned long flags; =20 + dept_key_init(&key->dkey); + if (WARN_ON_ONCE(static_obj(key))) return; hash_head =3D keyhashentry(key); @@ -4362,6 +4364,8 @@ static void __trace_hardirqs_on_caller(void) */ void lockdep_hardirqs_on_prepare(void) { + dept_hardirqs_on(); + if (unlikely(!debug_locks)) return; =20 @@ -4482,6 +4486,8 @@ EXPORT_SYMBOL_GPL(lockdep_hardirqs_on); */ void noinstr lockdep_hardirqs_off(unsigned long ip) { + dept_hardirqs_off(); + if (unlikely(!debug_locks)) return; =20 @@ -4526,6 +4532,8 @@ void lockdep_softirqs_on(unsigned long ip) { struct irqtrace_events *trace =3D ¤t->irqtrace; =20 + dept_softirqs_on_ip(ip); + if (unlikely(!lockdep_enabled())) return; =20 @@ -4564,6 +4572,8 @@ void lockdep_softirqs_on(unsigned long ip) */ void lockdep_softirqs_off(unsigned long ip) { + dept_softirqs_off(); + if (unlikely(!lockdep_enabled())) return; =20 @@ -4941,6 +4951,8 @@ void lockdep_init_map_type(struct lockdep_map *lock, = const char *name, { int i; =20 + ldt_init(&lock->dmap, &key->dkey, subclass, name); + for (i =3D 0; i < NR_LOCKDEP_CACHING_CLASSES; i++) lock->class_cache[i] =3D NULL; =20 @@ -5734,6 +5746,12 @@ void lock_set_class(struct lockdep_map *lock, const = char *name, { unsigned long flags; =20 + /* + * dept_map_(re)init() might be called twice redundantly. But + * there's no choice as long as Dept relies on Lockdep. + */ + ldt_set_class(&lock->dmap, name, &key->dkey, subclass, ip); + if (unlikely(!lockdep_enabled())) return; =20 @@ -5751,6 +5769,8 @@ void lock_downgrade(struct lockdep_map *lock, unsigne= d long ip) { unsigned long flags; =20 + ldt_downgrade(&lock->dmap, ip); + if (unlikely(!lockdep_enabled())) return; =20 @@ -6586,6 +6606,8 @@ void lockdep_unregister_key(struct lock_class_key *ke= y) bool found =3D false; bool need_callback =3D false; =20 + dept_key_destroy(&key->dkey); + might_sleep(); =20 if (WARN_ON_ONCE(static_obj(key))) --=20 2.17.1 From nobody Fri Dec 19 15:51:27 2025 Received: from invmail4.hynix.com (exvmail4.skhynix.com [166.125.252.92]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 26A8126FA7D; Mon, 19 May 2025 09:18:50 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=166.125.252.92 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747646335; cv=none; b=KvjR3a92K6mh6b+Mb+y+CY9bPvc9KkscPV1VUaAwUjDZ3WcTFVCEqa7S3kvb6jlkhO/5ukbJgbb5aD2k7LfXe1H1snaJRNeOzUOr1rHPXnxBJhU11779RUv9fANwSALWwHzNbecybUGUf6i7HzxfqYRsh2L8ewjC4yiI1lsUpEk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747646335; c=relaxed/simple; bh=7Twj65zRSkQ/WMNSTWEyxzgXy8V9jjNnIrH+Kg2KsWY=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References; b=ON9JHjC83PvtNb1OT/ohoMfAaOikwDo82AehnN/xYE4PyL8+ErXFIimZZmvpunHJaJCdABpIuy1zrA/E8sT0ak7Gty0UkmRmuLZWslZk1EiShDiz2Pl2f0u1m80o/vq9QjwA5fYhq5VsMfLwHYQtfOPVyBB41ttyqEAJOqTWb/4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com; spf=pass smtp.mailfrom=sk.com; arc=none smtp.client-ip=166.125.252.92 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=sk.com X-AuditID: a67dfc5b-669ff7000002311f-71-682af76ded55 From: Byungchul Park To: linux-kernel@vger.kernel.org Cc: kernel_team@skhynix.com, torvalds@linux-foundation.org, damien.lemoal@opensource.wdc.com, linux-ide@vger.kernel.org, adilger.kernel@dilger.ca, linux-ext4@vger.kernel.org, mingo@redhat.com, peterz@infradead.org, will@kernel.org, tglx@linutronix.de, rostedt@goodmis.org, joel@joelfernandes.org, sashal@kernel.org, daniel.vetter@ffwll.ch, duyuyang@gmail.com, johannes.berg@intel.com, tj@kernel.org, tytso@mit.edu, willy@infradead.org, david@fromorbit.com, amir73il@gmail.com, gregkh@linuxfoundation.org, kernel-team@lge.com, linux-mm@kvack.org, akpm@linux-foundation.org, mhocko@kernel.org, minchan@kernel.org, hannes@cmpxchg.org, vdavydov.dev@gmail.com, sj@kernel.org, jglisse@redhat.com, dennis@kernel.org, cl@linux.com, penberg@kernel.org, rientjes@google.com, vbabka@suse.cz, ngupta@vflare.org, linux-block@vger.kernel.org, josef@toxicpanda.com, linux-fsdevel@vger.kernel.org, jack@suse.cz, jlayton@kernel.org, dan.j.williams@intel.com, hch@infradead.org, djwong@kernel.org, dri-devel@lists.freedesktop.org, rodrigosiqueiramelo@gmail.com, melissa.srw@gmail.com, hamohammed.sa@gmail.com, harry.yoo@oracle.com, chris.p.wilson@intel.com, gwan-gyeong.mun@intel.com, max.byungchul.park@gmail.com, boqun.feng@gmail.com, longman@redhat.com, yskelg@gmail.com, yunseong.kim@ericsson.com, yeoreum.yun@arm.com, netdev@vger.kernel.org, matthew.brost@intel.com, her0gyugyu@gmail.com Subject: [PATCH v16 06/42] dept: add proc knobs to show stats and dependency graph Date: Mon, 19 May 2025 18:17:50 +0900 Message-Id: <20250519091826.19752-7-byungchul@sk.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20250519091826.19752-1-byungchul@sk.com> References: <20250519091826.19752-1-byungchul@sk.com> X-Brightmail-Tracker: H4sIAAAAAAAAAzWSa0wTaRSG/b6ZzgzFmkk1OoJR0mhQjJcaNeeHom7MOjGaGI0mYsxuXWZt tRTSIorJJtxEF4QoWlBALNVUaKvUTrNhd63ctIoXREXuoFSDS7gZpKVcdLfg+ufkSc77PufP YQj5Y0kYo9ElCnqdSqugpKR0cLZ5lW4sSr3W2rAUfKPnSCiusFPQeMeGwO5KxdD3cAe0+AcQ TD5/QUCBsRFBaU8XAS5PNwJ3WRoFrz/MgSbfMAX1xmwK0m9UUPCyfwpDZ34eBptzN7y19JLw 9IIZQ0EfBUUF6Tg4/sEwbrHSYElZBt6yQhqmepRQ390sAXf7Srha0knBPXc9CZ5KL4bXfxVT 0G3/VwJPPY9J8OeGQ+PFHAncHjJT0O+3EGDxDdPwqtqEwWOaD46MoDDz81cJPMqpxpB58y6G pra/Edw/9w6D095MQZ1vAIPoNBIwceshAm/uIA1nzo/TUJSaiyD7TD4JGZ0bYDIQvHxtVAmp 1x0k3P7SjLZu5u0ldsTXDQwTfIZ4kp/wvaF4t99E8k/MHP9nYRfNZ9xvp3mT8wQvlkXxN+71 Yb50xCfhndbfKd45kkfzWYNNmB9qaKD3LIqRbooVtJokQb8m+mepuuvyJZzQvfnU25xRMgWJ yiwUwnDses5vnpB857PvrNQ0U2wk19o6TkzzPDaCE3N6gxkpQ7DNoVzLtTY0vZjL7uXKJwMz BZJdxtnbxBmWsRs4e7WR/CZdwtkc1TOiEHYj155dN9OVBzNNthJyWsqxl0K4qkLx/8JCrqas lbyAZCY0y4rkGl1SnEqjXb9anazTnFr9S3ycEwX/y/Lb1KFKNNK4rxaxDFLMljncK9RyiSrJ kBxXiziGUMyTWcXlarksVpV8WtDH/6Q/oRUMtSicIRULZOv8J2Pl7FFVonBcEBIE/fctZkLC UlDms72r0hxFLluoJX6b3J2gjc4/XjN2ZQdV6X0fqPgq7t9VU/zj6DEcf7gva8Lj+mNx4Mum 58bYu/lVYWurajs2WiIi03e2TOUt+PWDN/HKwZiIB4aDWxb90PLIEdfzhDYmhXUc6CjoVaZ1 3XF9DHhXLE0bOvKprZw5cHXPx+2RR8YcCtKgVimjCL1B9R/1BxbOWwMAAA== X-Brightmail-Tracker: H4sIAAAAAAAAAzWSa0hTcRiH+5/rXC1Py+pglDGKytASsl4qLYj0FBaRH4S+5KhDG7opm7Ps IppmpmnXKVna1Ji2HWvuCGk5WQ41i9LSLGVaWUSWFyi38tJlGn15eeD3/p73yyvB5RYyUKLW pvA6rTJRQUkJ6b6tWSGaH8GqDUXWheAZzyXg5j2Bgs67VgRCXSYGQy3R8No7jGDqWQcOxcZO BOXv+3Goax1A4Kg+Q0HXx/nQ7RmjoN2YT0FW5T0KXnydxsBddAUDq30vvDV/IuDppQoMioco uFGchfnGZwwmzBYazBmrYLC6hIbp92HQPtBDgqu0nQRH3zq4XuamoNHRTkBr/SAGXQ9uUjAg /CHhaetjAryFS6HzcgEJNaMVFHz1mnEwe8ZoeOk0YdBqWgy2bJ815/tvEtoKnBjk3K7FoLv3 IYKm3HcY2IUeClyeYQxEuxGHyaoWBIOFIzScvTBBw43MQgT5Z4sIyHaHw9RP3+XS8TDIvGUj oOZXD9oRyQllAuJcw2M4ly0e4yY9ryjO4TUR3JMKlmso6ae57KY+mjPZDZxYHcxVNg5hXPk3 D8nZLecpzv7tCs3ljXRj3Ojz5/T+ZQel247wiepUXrc+Ml6q6r92FUseiDj+tmCcyEBiWB7y k7DMRvbcOws1wxSzmn3zZgKf4QBmBSsWfCLzkFSCMz1z2delvWgmWMgcYO9M/ZwtEMwqVugV Z1nGhLOC00j8kwaxVptzVuTHbGL78l2zXblvp9taRlxCUhOaY0EBam2qRqlODA/VJ6jStOrj oYeTNHbk+yDz6enL9Wi8K7oZMRKkmCezOdaq5KQyVZ+maUasBFcEyCziGpVcdkSZdoLXJR3S GRJ5fTNaKiEUS2R74vh4OXNUmcIn8Hwyr/ufYhK/wAykW8nb0h2VCXG3gpIexU7lXFjv5k+X HFywPKYxPajsTLh3ZLXx86iqTjRHzTVs2HY9F0/piHCbNevu676EvEyIE3ee2r65CtL9vbW7 IopWNvjHxBoqnPVaoU09SG8JlI1sDBrdHWk6OY3HaauzPkT1t100nCunJ6uOxSs1NYtcCkKv UoYF4zq98i+FbAMvPQMAAA== X-CFilter-Loop: Reflected Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" It'd be useful to show dept internal stats and dependency graph on runtime via proc for better information. Introduce the knobs. Signed-off-by: Byungchul Park --- kernel/dependency/Makefile | 1 + kernel/dependency/dept.c | 50 +++------------- kernel/dependency/dept_internal.h | 54 +++++++++++++++++ kernel/dependency/dept_proc.c | 96 +++++++++++++++++++++++++++++++ 4 files changed, 160 insertions(+), 41 deletions(-) create mode 100644 kernel/dependency/dept_internal.h create mode 100644 kernel/dependency/dept_proc.c diff --git a/kernel/dependency/Makefile b/kernel/dependency/Makefile index b5cfb8a03c0c..92f165400187 100644 --- a/kernel/dependency/Makefile +++ b/kernel/dependency/Makefile @@ -1,3 +1,4 @@ # SPDX-License-Identifier: GPL-2.0 =20 obj-$(CONFIG_DEPT) +=3D dept.o +obj-$(CONFIG_DEPT) +=3D dept_proc.o diff --git a/kernel/dependency/dept.c b/kernel/dependency/dept.c index 6cdda00411bc..9650d6d0c4d0 100644 --- a/kernel/dependency/dept.c +++ b/kernel/dependency/dept.c @@ -75,6 +75,7 @@ #include #include #include +#include "dept_internal.h" =20 static int dept_stop; static int dept_per_cpu_ready; @@ -265,46 +266,13 @@ static bool valid_key(struct dept_key *k) * have been freed will be placed. */ =20 -enum object_t { -#define OBJECT(id, nr) OBJECT_##id, - #include "dept_object.h" -#undef OBJECT - OBJECT_NR, -}; - #define OBJECT(id, nr) \ static struct dept_##id spool_##id[nr]; \ static DEFINE_PER_CPU(struct llist_head, lpool_##id); #include "dept_object.h" #undef OBJECT =20 -struct dept_pool { - const char *name; - - /* - * object size - */ - size_t obj_sz; - - /* - * the number of the static array - */ - atomic_t obj_nr; - - /* - * offset of ->pool_node - */ - size_t node_off; - - /* - * pointer to the pool - */ - void *spool; - struct llist_head boot_pool; - struct llist_head __percpu *lpool; -}; - -static struct dept_pool pool[OBJECT_NR] =3D { +struct dept_pool dept_pool[OBJECT_NR] =3D { #define OBJECT(id, nr) { \ .name =3D #id, \ .obj_sz =3D sizeof(struct dept_##id), \ @@ -334,7 +302,7 @@ static void *from_pool(enum object_t t) if (DEPT_WARN_ON(!irqs_disabled())) return NULL; =20 - p =3D &pool[t]; + p =3D &dept_pool[t]; =20 /* * Try local pool first. @@ -369,7 +337,7 @@ static void *from_pool(enum object_t t) =20 static void to_pool(void *o, enum object_t t) { - struct dept_pool *p =3D &pool[t]; + struct dept_pool *p =3D &dept_pool[t]; struct llist_head *h; =20 preempt_disable(); @@ -2108,7 +2076,7 @@ void dept_map_copy(struct dept_map *to, struct dept_m= ap *from) clean_classes_cache(&to->map_key); } =20 -static LIST_HEAD(classes); +LIST_HEAD(dept_classes); =20 static bool within(const void *addr, void *start, unsigned long size) { @@ -2140,7 +2108,7 @@ void dept_free_range(void *start, unsigned int sz) while (unlikely(!dept_lock())) cpu_relax(); =20 - list_for_each_entry_safe(c, n, &classes, all_node) { + list_for_each_entry_safe(c, n, &dept_classes, all_node) { if (!within((void *)c->key, start, sz) && !within(c->name, start, sz)) continue; @@ -2216,7 +2184,7 @@ static struct dept_class *check_new_class(struct dept= _key *local, c->sub_id =3D sub_id; c->key =3D (unsigned long)(k->base + sub_id); hash_add_class(c); - list_add(&c->all_node, &classes); + list_add(&c->all_node, &dept_classes); unlock: dept_unlock(); caching: @@ -2950,8 +2918,8 @@ static void migrate_per_cpu_pool(void) struct llist_head *from; struct llist_head *to; =20 - from =3D &pool[i].boot_pool; - to =3D per_cpu_ptr(pool[i].lpool, boot_cpu); + from =3D &dept_pool[i].boot_pool; + to =3D per_cpu_ptr(dept_pool[i].lpool, boot_cpu); move_llist(to, from); } } diff --git a/kernel/dependency/dept_internal.h b/kernel/dependency/dept_int= ernal.h new file mode 100644 index 000000000000..6b39e5a2a830 --- /dev/null +++ b/kernel/dependency/dept_internal.h @@ -0,0 +1,54 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* + * Dept(DEPendency Tracker) - runtime dependency tracker internal header + * + * Started by Byungchul Park : + * + * Copyright (c) 2020 LG Electronics, Inc., Byungchul Park + * Copyright (c) 2024 SK hynix, Inc., Byungchul Park + */ + +#ifndef __DEPT_INTERNAL_H +#define __DEPT_INTERNAL_H + +#ifdef CONFIG_DEPT +#include + +struct dept_pool { + const char *name; + + /* + * object size + */ + size_t obj_sz; + + /* + * the number of the static array + */ + atomic_t obj_nr; + + /* + * offset of ->pool_node + */ + size_t node_off; + + /* + * pointer to the pool + */ + void *spool; + struct llist_head boot_pool; + struct llist_head __percpu *lpool; +}; + +enum object_t { +#define OBJECT(id, nr) OBJECT_##id, + #include "dept_object.h" +#undef OBJECT + OBJECT_NR, +}; + +extern struct list_head dept_classes; +extern struct dept_pool dept_pool[]; + +#endif +#endif /* __DEPT_INTERNAL_H */ diff --git a/kernel/dependency/dept_proc.c b/kernel/dependency/dept_proc.c new file mode 100644 index 000000000000..97beaf397715 --- /dev/null +++ b/kernel/dependency/dept_proc.c @@ -0,0 +1,96 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * Procfs knobs for Dept(DEPendency Tracker) + * + * Started by Byungchul Park : + * + * Copyright (C) 2021 LG Electronics, Inc. , Byungchul Park + * Copyright (C) 2024 SK hynix, Inc. , Byungchul Park + */ +#include +#include +#include +#include "dept_internal.h" + +static void *l_next(struct seq_file *m, void *v, loff_t *pos) +{ + /* + * XXX: Serialize list traversal if needed. The following might + * give a wrong information on contention. + */ + return seq_list_next(v, &dept_classes, pos); +} + +static void *l_start(struct seq_file *m, loff_t *pos) +{ + /* + * XXX: Serialize list traversal if needed. The following might + * give a wrong information on contention. + */ + return seq_list_start_head(&dept_classes, *pos); +} + +static void l_stop(struct seq_file *m, void *v) +{ +} + +static int l_show(struct seq_file *m, void *v) +{ + struct dept_class *fc =3D list_entry(v, struct dept_class, all_node); + struct dept_dep *d; + const char *prefix; + + if (v =3D=3D &dept_classes) { + seq_puts(m, "All classes:\n\n"); + return 0; + } + + prefix =3D fc->sched_map ? " " : ""; + seq_printf(m, "[%p] %s%s\n", (void *)fc->key, prefix, fc->name); + + /* + * XXX: Serialize list traversal if needed. The following might + * give a wrong information on contention. + */ + list_for_each_entry(d, &fc->dep_head, dep_node) { + struct dept_class *tc =3D d->wait->class; + + prefix =3D tc->sched_map ? " " : ""; + seq_printf(m, " -> [%p] %s%s\n", (void *)tc->key, prefix, tc->name); + } + seq_puts(m, "\n"); + + return 0; +} + +static const struct seq_operations dept_deps_ops =3D { + .start =3D l_start, + .next =3D l_next, + .stop =3D l_stop, + .show =3D l_show, +}; + +static int dept_stats_show(struct seq_file *m, void *v) +{ + int r; + + seq_puts(m, "Availability in the static pools:\n\n"); +#define OBJECT(id, nr) \ + r =3D atomic_read(&dept_pool[OBJECT_##id].obj_nr); \ + if (r < 0) \ + r =3D 0; \ + seq_printf(m, "%s\t%d/%d(%d%%)\n", #id, r, nr, (r * 100) / (nr)); + #include "dept_object.h" +#undef OBJECT + + return 0; +} + +static int __init dept_proc_init(void) +{ + proc_create_seq("dept_deps", S_IRUSR, NULL, &dept_deps_ops); + proc_create_single("dept_stats", S_IRUSR, NULL, dept_stats_show); + return 0; +} + +__initcall(dept_proc_init); --=20 2.17.1 From nobody Fri Dec 19 15:51:27 2025 Received: from invmail4.hynix.com (exvmail4.hynix.com [166.125.252.92]) by smtp.subspace.kernel.org (Postfix) with ESMTP id A8EA826F45D; Mon, 19 May 2025 09:18:50 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=166.125.252.92 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747646334; cv=none; b=DjlMj1gjnEI3AWsTmaSndmg/KaWUQsivo5e6FGvA0/zaZGjZJ2FJ1lrPeWk0oGPy4ATB6v6g7dcnqzdVVyS40V4Z+Br8QcmKxRiZuUxBJCjtjwDEdLcPT9CkhhOmRu+dYXsA/uPzHYFgirLib+8bacFNSFHACgSEvieCegsg0Eo= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747646334; c=relaxed/simple; bh=ezEJxJ4N1Vjrb6q0DqS69gfOLERcwARBMwf+UHXwu2M=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References; b=Am52ys0HAUUeLVuCBkgySE2fFhYzU0vIChs/5kWok3pHfB7aBxBVtcamF6QNAKjFdJirvcdu2pb5QH35QpfatRM/WP1dW9rfMhpzbYg/PDBq8W6fGzpK4moem6AjDAR42UZ5HIJCfeIl+VHWbfDUssgIqM3v7aFXFBsavRQV8uk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com; spf=pass smtp.mailfrom=sk.com; arc=none smtp.client-ip=166.125.252.92 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=sk.com X-AuditID: a67dfc5b-669ff7000002311f-72-682af76db36f From: Byungchul Park To: linux-kernel@vger.kernel.org Cc: kernel_team@skhynix.com, torvalds@linux-foundation.org, damien.lemoal@opensource.wdc.com, linux-ide@vger.kernel.org, adilger.kernel@dilger.ca, linux-ext4@vger.kernel.org, mingo@redhat.com, peterz@infradead.org, will@kernel.org, tglx@linutronix.de, rostedt@goodmis.org, joel@joelfernandes.org, sashal@kernel.org, daniel.vetter@ffwll.ch, duyuyang@gmail.com, johannes.berg@intel.com, tj@kernel.org, tytso@mit.edu, willy@infradead.org, david@fromorbit.com, amir73il@gmail.com, gregkh@linuxfoundation.org, kernel-team@lge.com, linux-mm@kvack.org, akpm@linux-foundation.org, mhocko@kernel.org, minchan@kernel.org, hannes@cmpxchg.org, vdavydov.dev@gmail.com, sj@kernel.org, jglisse@redhat.com, dennis@kernel.org, cl@linux.com, penberg@kernel.org, rientjes@google.com, vbabka@suse.cz, ngupta@vflare.org, linux-block@vger.kernel.org, josef@toxicpanda.com, linux-fsdevel@vger.kernel.org, jack@suse.cz, jlayton@kernel.org, dan.j.williams@intel.com, hch@infradead.org, djwong@kernel.org, dri-devel@lists.freedesktop.org, rodrigosiqueiramelo@gmail.com, melissa.srw@gmail.com, hamohammed.sa@gmail.com, harry.yoo@oracle.com, chris.p.wilson@intel.com, gwan-gyeong.mun@intel.com, max.byungchul.park@gmail.com, boqun.feng@gmail.com, longman@redhat.com, yskelg@gmail.com, yunseong.kim@ericsson.com, yeoreum.yun@arm.com, netdev@vger.kernel.org, matthew.brost@intel.com, her0gyugyu@gmail.com Subject: [PATCH v16 07/42] dept: distinguish each kernel context from another Date: Mon, 19 May 2025 18:17:51 +0900 Message-Id: <20250519091826.19752-8-byungchul@sk.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20250519091826.19752-1-byungchul@sk.com> References: <20250519091826.19752-1-byungchul@sk.com> X-Brightmail-Tracker: H4sIAAAAAAAAAzWSe0hTcRTH+917d+91ubosqZsW1SgsI82oOEQvev7CjKCgKKhGXttqLplT M4p8rJemmWaWpU6LtdystfWwx8QcLe1ptaaJWg2JLM2wtjLtsRn9c/hwzvl+zj+HJaVOUSir VGsFjVquktFiStwTXDFD/T1CMfPNk8ng/XaEgnNXzDQ0XTYhMF/LIKDr/kpo9nUjGHjyjITi oiYEFe/aSbjm7EBgN2bS8LJzBLi8vTQ0FuXQkHX+Cg3PPw0S0HaqgACTNRbeGN5T8Ci/koDi LhrOFmcR/vKBgH5DFQOG9CngMZYwMPguGho73CKwt06HM2VtNNy1N1LgrPEQ8PL2ORo6zH9E 8MjZQIEvLwyaTuSKoPpzJQ2ffAYSDN5eBl7U6Qlw6keDRecXHvr6WwQPcusIOHThKgGu13cQ 1B55S4DV7KbB4e0mwGYtIuHnxfsIPHk9DBw81s/A2Yw8BDkHT1Gga5sDAz/8l0u/RUNGuYWC 6l9utHgBNpeZEXZ095JYZ0vFP72vaGz36Sn8sJLHt0raGayrbWWw3pqMbcYIfP5uF4Er+rwi bK06SmNrXwGDs3tcBP789Cmzdtwm8fw4QaVMETRRC7eJFQ0nG1CiZ8Oex+0NVDoyLM9GLMtz s/kL7nnZKGgIB/JviAJMc+F8S0s/GeAQbiJvy33v74tZknMP55tLX6PAYBQXy7sKTzMBprgp vLncgQJOCTeHf2YO/+ecwJssdUOeIG4u35rjGIpK/SsuUxkVcPJcYRDvzCxm/gXG8veMLVQ+ kujRsCokVapTEuRK1exIRZpauSdy++4EK/J/l2H/4OYa1Ne0rh5xLJIFSyz2aQqpSJ6SlJZQ j3iWlIVIqmxTFVJJnDxtr6DZvVWTrBKS6lEYS8nGSGb5UuOk3A65VtglCImC5v+UYINC01Hq ogk2n1a9M/56WX/nhkSNdlRJNbk+9nnCmGFR5Z59Mc13jodmhU4q7DNmZ4aJTVgTUj1pTVTP wqX8sr01x1eJU0K6Hmg35UQfKKiN/xIvhC/5fmNycGvn6OX5y3QbD9SZ1LVrPl7fH+n48Wvd pW0jtTGWm/terTYd/q0av2JL6erkcBmVpJBHR5CaJPlfuEKuxlkDAAA= X-Brightmail-Tracker: H4sIAAAAAAAAAzWSV0xTcRTG/d9NteZaidyARlNDVIxiI5gTJYTEdWOC8cGR+EKLXm1DqaZV FFcYtaIMEUUUBcpIZVwEb4lx1RAqCC7QVpaAiiMSGQZpFcFRNL6c/PJ953zfy2FwRQUZyOgM BwSjQaNXUjJCtnlN6rL4byHaFa11c8EzlkbA1RqRgrbrVQjEumQMBho3Qod3EMHE01Yc8nLb EBS/7cWhrqkPgaM8hQLX+5ng9oxQ0JKbTkFqaQ0Fzz9PYtBzMQeDKikaXts+EvA4uwSDvAEK ruSlYr7xCYNxWyUNtqRg6C/Pp2HyrQpa+tpJcBa0kODoXgqXC3souOdoIaDpVj8GrjtXKegT f5PwuKmZAG9WELSdyySheriEgs9eGw42zwgNL+qtGDRZ50Ct2Zdq+fqLhIeZ9RhYym5g4O66 i+B+2hsMJLGdAqdnEAO7lIvDj2uNCPqzhmg4mTFOw5XkLATpJy8SYO4Jh4nvvuaCMRUkF9US UP2zHUVF8mKhiHjn4AjOm+2H+B+elxTv8FoJ/lEJx9/O76V58/1umrdKB3l7eQhfem8A44tH PSQvVZ6meGk0h+bPDLkxfvjZM3rLvJ2yiN2CXpcgGEMj1TJt84VmtL9/x+Envc1EErKtP4P8 GI4N4yayb5JTTLGLuM7OcXyK/dkFnD3zo0+XMTjbPp3rKOhCU8ZsNppzn79ETzHBBnNikdOn M4ycDedaxUX/MudzVbX1f3P82FVcd7rz76nCt+KuKiSykcyKplUif50hIV6j04cvN8VpEw26 w8t37YuXkO+BbMcnz91CY66NDYhlkHKGvNaxRKsgNQmmxPgGxDG40l9eaV+sVch3axKPCMZ9 McaDesHUgIIYQhkg37RDUCvYvZoDQpwg7BeM/12M8QtMQnvUaTEe85wnsWmSxSy3lE7m39i+ LeOX9WxdhevEl3dhxyLI7HVBLpPo7iRnFOsbr63Ni1GOV8x6LwVPW/vgza7qC9HDq83qDu9Y oX9qVGi5mKFQLDVsPhWzdWWKOor5UOQOWxcaVLYh67pS9QoC1jwoUB2NK4u1dOVEBlhbF0Yo CZNWowrBjSbNH8v9LsA8AwAA X-CFilter-Loop: Reflected Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Each unique kernel context, in dept's point of view, should be identified on every entrance to kernel mode e.g. system call or user oriented fault. Otherwise, dept may track meaningless dependencies across different kernel context. Plus, in order to update kernel context id at the very beginning of each entrance, arch code support is required, that could be configured by CONFIG_ARCH_HAS_DEPT_SUPPORT. Signed-off-by: Byungchul Park --- include/linux/dept.h | 29 ++++++++++------- include/linux/sched.h | 10 +++--- kernel/dependency/dept.c | 67 ++++++++++++++++++++-------------------- lib/Kconfig.debug | 5 ++- 4 files changed, 61 insertions(+), 50 deletions(-) diff --git a/include/linux/dept.h b/include/linux/dept.h index 5f0d2d8c8cbe..cb1b1beea077 100644 --- a/include/linux/dept.h +++ b/include/linux/dept.h @@ -26,11 +26,16 @@ struct task_struct; #define DEPT_MAX_SUBCLASSES_USR (DEPT_MAX_SUBCLASSES / DEPT_MAX_SUBCLASSE= S_EVT) #define DEPT_MAX_SUBCLASSES_CACHE 2 =20 -#define DEPT_SIRQ 0 -#define DEPT_HIRQ 1 -#define DEPT_IRQS_NR 2 -#define DEPT_SIRQF (1UL << DEPT_SIRQ) -#define DEPT_HIRQF (1UL << DEPT_HIRQ) +enum { + DEPT_CXT_SIRQ =3D 0, + DEPT_CXT_HIRQ, + DEPT_CXT_IRQS_NR, + DEPT_CXT_PROCESS =3D DEPT_CXT_IRQS_NR, + DEPT_CXTS_NR +}; + +#define DEPT_SIRQF (1UL << DEPT_CXT_SIRQ) +#define DEPT_HIRQF (1UL << DEPT_CXT_HIRQ) =20 struct dept_ecxt; struct dept_iecxt { @@ -95,8 +100,8 @@ struct dept_class { /* * for tracking IRQ dependencies */ - struct dept_iecxt iecxt[DEPT_IRQS_NR]; - struct dept_iwait iwait[DEPT_IRQS_NR]; + struct dept_iecxt iecxt[DEPT_CXT_IRQS_NR]; + struct dept_iwait iwait[DEPT_CXT_IRQS_NR]; =20 /* * classified by a map embedded in task_struct, @@ -208,8 +213,8 @@ struct dept_ecxt { /* * where the IRQ-enabled happened */ - unsigned long enirq_ip[DEPT_IRQS_NR]; - struct dept_stack *enirq_stack[DEPT_IRQS_NR]; + unsigned long enirq_ip[DEPT_CXT_IRQS_NR]; + struct dept_stack *enirq_stack[DEPT_CXT_IRQS_NR]; =20 /* * where the event context started @@ -253,8 +258,8 @@ struct dept_wait { /* * where the IRQ wait happened */ - unsigned long irq_ip[DEPT_IRQS_NR]; - struct dept_stack *irq_stack[DEPT_IRQS_NR]; + unsigned long irq_ip[DEPT_CXT_IRQS_NR]; + struct dept_stack *irq_stack[DEPT_CXT_IRQS_NR]; =20 /* * where the wait happened @@ -384,6 +389,7 @@ extern void dept_event(struct dept_map *m, unsigned lon= g e_f, unsigned long ip, extern void dept_ecxt_exit(struct dept_map *m, unsigned long e_f, unsigned= long ip); extern void dept_sched_enter(void); extern void dept_sched_exit(void); +extern void dept_update_cxt(void); =20 static inline void dept_ecxt_enter_nokeep(struct dept_map *m) { @@ -431,6 +437,7 @@ struct dept_map { }; #define dept_ecxt_exit(m, e_f, ip) do { } while (0) #define dept_sched_enter() do { } while (0) #define dept_sched_exit() do { } while (0) +#define dept_update_cxt() do { } while (0) #define dept_ecxt_enter_nokeep(m) do { } while (0) #define dept_key_init(k) do { (void)(k); } while (0) #define dept_key_destroy(k) do { (void)(k); } while (0) diff --git a/include/linux/sched.h b/include/linux/sched.h index a1924b40feb5..802fca4d99b3 100644 --- a/include/linux/sched.h +++ b/include/linux/sched.h @@ -827,19 +827,19 @@ struct dept_task { int wait_hist_pos; =20 /* - * sequential id to identify each IRQ context + * sequential id to identify each context */ - unsigned int irq_id[DEPT_IRQS_NR]; + unsigned int cxt_id[DEPT_CXTS_NR]; =20 /* * for tracking IRQ-enabled points with cross-event */ - unsigned int wgen_enirq[DEPT_IRQS_NR]; + unsigned int wgen_enirq[DEPT_CXT_IRQS_NR]; =20 /* * for keeping up-to-date IRQ-enabled points */ - unsigned long enirq_ip[DEPT_IRQS_NR]; + unsigned long enirq_ip[DEPT_CXT_IRQS_NR]; =20 /* * for reserving a current stack instance at each operation @@ -893,7 +893,7 @@ struct dept_task { .wait_hist =3D { { .wait =3D NULL, } }, \ .ecxt_held_pos =3D 0, \ .wait_hist_pos =3D 0, \ - .irq_id =3D { 0U }, \ + .cxt_id =3D { 0U }, \ .wgen_enirq =3D { 0U }, \ .enirq_ip =3D { 0UL }, \ .stack =3D NULL, \ diff --git a/kernel/dependency/dept.c b/kernel/dependency/dept.c index 9650d6d0c4d0..b4a39c81bbc1 100644 --- a/kernel/dependency/dept.c +++ b/kernel/dependency/dept.c @@ -230,9 +230,9 @@ static struct dept_class *dep_tc(struct dept_dep *d) =20 static const char *irq_str(int irq) { - if (irq =3D=3D DEPT_SIRQ) + if (irq =3D=3D DEPT_CXT_SIRQ) return "softirq"; - if (irq =3D=3D DEPT_HIRQ) + if (irq =3D=3D DEPT_CXT_HIRQ) return "hardirq"; return "(unknown)"; } @@ -410,7 +410,7 @@ static void initialize_class(struct dept_class *c) { int i; =20 - for (i =3D 0; i < DEPT_IRQS_NR; i++) { + for (i =3D 0; i < DEPT_CXT_IRQS_NR; i++) { struct dept_iecxt *ie =3D &c->iecxt[i]; struct dept_iwait *iw =3D &c->iwait[i]; =20 @@ -436,7 +436,7 @@ static void initialize_ecxt(struct dept_ecxt *e) { int i; =20 - for (i =3D 0; i < DEPT_IRQS_NR; i++) { + for (i =3D 0; i < DEPT_CXT_IRQS_NR; i++) { e->enirq_stack[i] =3D NULL; e->enirq_ip[i] =3D 0UL; } @@ -452,7 +452,7 @@ static void initialize_wait(struct dept_wait *w) { int i; =20 - for (i =3D 0; i < DEPT_IRQS_NR; i++) { + for (i =3D 0; i < DEPT_CXT_IRQS_NR; i++) { w->irq_stack[i] =3D NULL; w->irq_ip[i] =3D 0UL; } @@ -491,7 +491,7 @@ static void destroy_ecxt(struct dept_ecxt *e) { int i; =20 - for (i =3D 0; i < DEPT_IRQS_NR; i++) + for (i =3D 0; i < DEPT_CXT_IRQS_NR; i++) if (e->enirq_stack[i]) put_stack(e->enirq_stack[i]); if (e->class) @@ -507,7 +507,7 @@ static void destroy_wait(struct dept_wait *w) { int i; =20 - for (i =3D 0; i < DEPT_IRQS_NR; i++) + for (i =3D 0; i < DEPT_CXT_IRQS_NR; i++) if (w->irq_stack[i]) put_stack(w->irq_stack[i]); if (w->class) @@ -665,7 +665,7 @@ static void print_diagram(struct dept_dep *d) const char *tc_n =3D tc->sched_map ? "" : (tc->name ?: "(unknown)"= ); =20 irqf =3D e->enirqf & w->irqf; - for_each_set_bit(irq, &irqf, DEPT_IRQS_NR) { + for_each_set_bit(irq, &irqf, DEPT_CXT_IRQS_NR) { if (!firstline) pr_warn("\nor\n\n"); firstline =3D false; @@ -698,7 +698,7 @@ static void print_dep(struct dept_dep *d) const char *tc_n =3D tc->sched_map ? "" : (tc->name ?: "(unknown)"= ); =20 irqf =3D e->enirqf & w->irqf; - for_each_set_bit(irq, &irqf, DEPT_IRQS_NR) { + for_each_set_bit(irq, &irqf, DEPT_CXT_IRQS_NR) { pr_warn("%s has been enabled:\n", irq_str(irq)); print_ip_stack(e->enirq_ip[irq], e->enirq_stack[irq]); pr_warn("\n"); @@ -866,7 +866,7 @@ static void bfs(void *root, struct bfs_ops *ops, void *= in, void **out) */ =20 static unsigned long cur_enirqf(void); -static int cur_irq(void); +static int cur_cxt(void); static unsigned int cur_ctxt_id(void); =20 static struct dept_iecxt *iecxt(struct dept_class *c, int irq) @@ -1443,7 +1443,7 @@ static void add_dep(struct dept_ecxt *e, struct dept_= wait *w) if (d) { check_dl_bfs(d); =20 - for (i =3D 0; i < DEPT_IRQS_NR; i++) { + for (i =3D 0; i < DEPT_CXT_IRQS_NR; i++) { struct dept_iwait *fiw =3D iwait(fc, i); struct dept_iecxt *found_ie; struct dept_iwait *found_iw; @@ -1487,7 +1487,7 @@ static void add_wait(struct dept_class *c, unsigned l= ong ip, struct dept_task *dt =3D dept_task(); struct dept_wait *w; unsigned int wg; - int irq; + int cxt; int i; =20 if (DEPT_WARN_ON(!valid_class(c))) @@ -1503,9 +1503,9 @@ static void add_wait(struct dept_class *c, unsigned l= ong ip, w->wait_stack =3D get_current_stack(); w->sched_sleep =3D sched_sleep; =20 - irq =3D cur_irq(); - if (irq < DEPT_IRQS_NR) - add_iwait(c, irq, w); + cxt =3D cur_cxt(); + if (cxt =3D=3D DEPT_CXT_HIRQ || cxt =3D=3D DEPT_CXT_SIRQ) + add_iwait(c, cxt, w); =20 /* * Avoid adding dependency between user aware nested ecxt and @@ -1579,7 +1579,7 @@ static struct dept_ecxt_held *add_ecxt(struct dept_ma= p *m, eh->sub_l =3D sub_l; =20 irqf =3D cur_enirqf(); - for_each_set_bit(irq, &irqf, DEPT_IRQS_NR) + for_each_set_bit(irq, &irqf, DEPT_CXT_IRQS_NR) add_iecxt(c, irq, e, false); =20 del_ecxt(e); @@ -1728,7 +1728,7 @@ static void do_event(struct dept_map *m, struct dept_= map *real_m, add_dep(eh->ecxt, wh->wait); } =20 - for (i =3D 0; i < DEPT_IRQS_NR; i++) { + for (i =3D 0; i < DEPT_CXT_IRQS_NR; i++) { struct dept_ecxt *e; =20 if (before(dt->wgen_enirq[i], wg)) @@ -1775,7 +1775,7 @@ static void disconnect_class(struct dept_class *c) call_rcu(&d->rh, del_dep_rcu); } =20 - for (i =3D 0; i < DEPT_IRQS_NR; i++) { + for (i =3D 0; i < DEPT_CXT_IRQS_NR; i++) { stale_iecxt(iecxt(c, i)); stale_iwait(iwait(c, i)); } @@ -1800,27 +1800,21 @@ static unsigned long cur_enirqf(void) return 0UL; } =20 -static int cur_irq(void) +static int cur_cxt(void) { if (lockdep_softirq_context(current)) - return DEPT_SIRQ; + return DEPT_CXT_SIRQ; if (lockdep_hardirq_context()) - return DEPT_HIRQ; - return DEPT_IRQS_NR; + return DEPT_CXT_HIRQ; + return DEPT_CXT_PROCESS; } =20 static unsigned int cur_ctxt_id(void) { struct dept_task *dt =3D dept_task(); - int irq =3D cur_irq(); + int cxt =3D cur_cxt(); =20 - /* - * Normal process context - */ - if (irq =3D=3D DEPT_IRQS_NR) - return 0U; - - return dt->irq_id[irq] | (1UL << irq); + return dt->cxt_id[cxt] | (1UL << cxt); } =20 static void enirq_transition(int irq) @@ -1877,7 +1871,7 @@ static void dept_enirq(unsigned long ip) =20 flags =3D dept_enter(); =20 - for_each_set_bit(irq, &irqf, DEPT_IRQS_NR) { + for_each_set_bit(irq, &irqf, DEPT_CXT_IRQS_NR) { dt->enirq_ip[irq] =3D ip; enirq_transition(irq); } @@ -1923,6 +1917,13 @@ void noinstr dept_hardirqs_off(void) dept_task()->hardirqs_enabled =3D false; } =20 +void noinstr dept_update_cxt(void) +{ + struct dept_task *dt =3D dept_task(); + + dt->cxt_id[DEPT_CXT_PROCESS] +=3D 1UL << DEPT_CXTS_NR; +} + /* * Ensure it's the outmost softirq context. */ @@ -1930,7 +1931,7 @@ void dept_softirq_enter(void) { struct dept_task *dt =3D dept_task(); =20 - dt->irq_id[DEPT_SIRQ] +=3D 1UL << DEPT_IRQS_NR; + dt->cxt_id[DEPT_CXT_SIRQ] +=3D 1UL << DEPT_CXTS_NR; } =20 /* @@ -1940,7 +1941,7 @@ void noinstr dept_hardirq_enter(void) { struct dept_task *dt =3D dept_task(); =20 - dt->irq_id[DEPT_HIRQ] +=3D 1UL << DEPT_IRQS_NR; + dt->cxt_id[DEPT_CXT_HIRQ] +=3D 1UL << DEPT_CXTS_NR; } =20 void dept_sched_enter(void) diff --git a/lib/Kconfig.debug b/lib/Kconfig.debug index 54b03db59a12..195c11b92bcf 100644 --- a/lib/Kconfig.debug +++ b/lib/Kconfig.debug @@ -1365,9 +1365,12 @@ config DEBUG_PREEMPT =20 menu "Lock Debugging (spinlocks, mutexes, etc...)" =20 +config ARCH_HAS_DEPT_SUPPORT + bool + config DEPT bool "Dependency tracking (EXPERIMENTAL)" - depends on DEBUG_KERNEL && LOCK_DEBUGGING_SUPPORT + depends on DEBUG_KERNEL && LOCK_DEBUGGING_SUPPORT && ARCH_HAS_DEPT_SUPPORT select DEBUG_SPINLOCK select DEBUG_MUTEXES if !PREEMPT_RT select DEBUG_RT_MUTEXES if RT_MUTEXES --=20 2.17.1 From nobody Fri Dec 19 15:51:27 2025 Received: from invmail4.hynix.com (exvmail4.hynix.com [166.125.252.92]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 32C2826A1AD; Mon, 19 May 2025 09:18:44 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=166.125.252.92 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747646329; cv=none; b=LUPqEP8zpalC9KGzo/0tQqjhC65Fa7eBPDsz35yMBLZUENElNYZiKp2osDsMQjUla/TbRgDUJwnK3jHkCdE6QnSBB3X4BSLpzLrORdMds4OrxGIaKF6DjJelLAXp4D4RCwMdoqj215RfF3I8rnQVsx33F5parOhMx8hefLtsYqE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747646329; c=relaxed/simple; bh=8SxiXT7aMr/Jndlc0Cvu8YV+kxqp+XaubTMN75I/AOI=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References; b=GMZwWm5jBqcUxkHqOQ4FjEqQZc5YqdqZ+E6EVtbBx9RKuWSRHZF4rK4FO7CaXpboP52ix0kxvRZmzIO33Wo2VnAP75MHMLcMkSRYZp71w+KeL3xbbrBf5nZ7OigPTJcQY6HD5vayBi69rQvToxRXPrrPiwwRNkfjlVlgGnntTPs= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com; spf=pass smtp.mailfrom=sk.com; arc=none smtp.client-ip=166.125.252.92 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=sk.com X-AuditID: a67dfc5b-669ff7000002311f-73-682af76d8185 From: Byungchul Park To: linux-kernel@vger.kernel.org Cc: kernel_team@skhynix.com, torvalds@linux-foundation.org, damien.lemoal@opensource.wdc.com, linux-ide@vger.kernel.org, adilger.kernel@dilger.ca, linux-ext4@vger.kernel.org, mingo@redhat.com, peterz@infradead.org, will@kernel.org, tglx@linutronix.de, rostedt@goodmis.org, joel@joelfernandes.org, sashal@kernel.org, daniel.vetter@ffwll.ch, duyuyang@gmail.com, johannes.berg@intel.com, tj@kernel.org, tytso@mit.edu, willy@infradead.org, david@fromorbit.com, amir73il@gmail.com, gregkh@linuxfoundation.org, kernel-team@lge.com, linux-mm@kvack.org, akpm@linux-foundation.org, mhocko@kernel.org, minchan@kernel.org, hannes@cmpxchg.org, vdavydov.dev@gmail.com, sj@kernel.org, jglisse@redhat.com, dennis@kernel.org, cl@linux.com, penberg@kernel.org, rientjes@google.com, vbabka@suse.cz, ngupta@vflare.org, linux-block@vger.kernel.org, josef@toxicpanda.com, linux-fsdevel@vger.kernel.org, jack@suse.cz, jlayton@kernel.org, dan.j.williams@intel.com, hch@infradead.org, djwong@kernel.org, dri-devel@lists.freedesktop.org, rodrigosiqueiramelo@gmail.com, melissa.srw@gmail.com, hamohammed.sa@gmail.com, harry.yoo@oracle.com, chris.p.wilson@intel.com, gwan-gyeong.mun@intel.com, max.byungchul.park@gmail.com, boqun.feng@gmail.com, longman@redhat.com, yskelg@gmail.com, yunseong.kim@ericsson.com, yeoreum.yun@arm.com, netdev@vger.kernel.org, matthew.brost@intel.com, her0gyugyu@gmail.com Subject: [PATCH v16 08/42] x86_64, dept: add support CONFIG_ARCH_HAS_DEPT_SUPPORT to x86_64 Date: Mon, 19 May 2025 18:17:52 +0900 Message-Id: <20250519091826.19752-9-byungchul@sk.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20250519091826.19752-1-byungchul@sk.com> References: <20250519091826.19752-1-byungchul@sk.com> X-Brightmail-Tracker: H4sIAAAAAAAAAzWSb0xTZxSH97739t7bSpebSuadLNvSxFRrhkLEnMw5TUzmm0zjEk00Ev80 cmObQTFFUEgWYVCtIuhMABVhBbFWKIKtJk4pKSWWgRlUqaUgVmXE2EklA1pWQVyr8cvJk/x+ 5zlfDkcpPJKlnE5/WDToNdlKRkbLwkkN3+hn1drVs11KiMyYaLjUZmPAe70Fge1mCYbQvc0w FJ1AMPfXAAU1VV4EDc+fUHDTE0TgtP7KwOD4p+CLTDLQW1XOQOnlNgYevJrHMFp9DkOLfSs8 tbyg4f7ZRgw1IQZqa0pxfLzEELM0s2ApXgZj1osszD9Pg96gXwLOkZVwoX6UgQ5nLw2e22MY Bu9cYiBoeyeB+54/aYhWpoD3twoJtL5uZOBV1EKBJTLJwkOXGYPH/Bm0l8WFx6cXJNBT4cJw vOkGBt/wXQSdpmcY7DY/A92RCQwOexUFb67eQzBWGWbBeDrGQm1JJYJyYzUNZaMZMPdf/HLd TBqU/N5OQ+tbP9q4ntjqbYh0T0xSpMxxhLyJPGKIM2qmSV+jQP64+IQlZZ0jLDHb84nDqiaX O0KYNExFJMTefJIh9qlzLDkV9mHyur+f/emL3bLvssRsXYFoWPX9fpn2cV2YPtSXfHSu9Apd jP7hTyEpJ/BrhPMNLslHDnTcYhPM8CohEIhRCU7mvxYcFS/iHRlH8f5FwlDdMEoEi/lM4d9b 8/GA42h+mWAqwQmU8xnCeFfBB+VXQku7671Gyq8VRsq7328q4hVfSz2dUAr8Gakw3XkBf1j4 XOiyBuizSG5GnzQjhU5fkKPRZa9J1RbqdUdTD+Tm2FH8uyy/zGfeRlPe7W7Ec0iZJG93rtAq JJqCvMIcNxI4Spksb3Ys1yrkWZrCItGQu8+Qny3muVEKRyuXyNOjR7IU/EHNYfFnUTwkGj6m mJMuLUY6FzUrS43tcI+GTE2nxSzP32jymtmY2ipN2WnsH3DvUjfe+DbT7lNdq1ZvNIV+2BqM 3Vmxtzpj3Vy4bZO2SMVsf7DamGRUWf2Ltjg3BGYGVEXnmR/3kKlNV4bagmq+78uFLdu8uTsX Tkw/Xb9uW/2CkIZ7nD3p47XHBnbLm04q6TytJk1NGfI0/wNg7ldfWQMAAA== X-Brightmail-Tracker: H4sIAAAAAAAAAzWSf0zMcRjHfb6/O46vc+M7MXaWCEemPBuzbI3PGGNjNjYcfeeOLnbHKWMr V9YP3bBVox+u4pw67rrL5kdnqTmSH9FJkqhhohPqjroLF/PPs9f2ej/v55+HI2VWeiqnSTko 6lJUyQpGQknWLzMu0P6IUS9yZSvAP5hNQYndxkDL1WoEttoMAnrvroYXgT4EwUdPSCgqaEFQ 3v2ahFpPFwK39TgDre/Gg9ffz0BTQR4Dxko7A08/hwjoLDxDQLVzHbyxfKCg+VQFAUW9DBQX GYnw+EjAkKWKBUt6FPRYz7EQ6o6Fpq42GhpLm2hwd8yDs2WdDNS5myjwXO8hoPVmCQNdtt80 NHvuUxAwRULL6XwarnypYOBzwEKCxd/PwrN6MwEe82RwZIZbTwz8ouFefj0BJy7UEOB9eQvB 7ey3BDhtbQw0+vsIcDkLSBi+dBdBj8nHQtbJIRaKM0wI8rIKKcjsjIPgz/Dl0sFYyDjvoODK SBtKWIFtZTaEG/v6SZzpOoyH/c8Z7A6YKfygQsA3zr1mcebtDhabnYewyxqDK+t6CVz+3U9j Z1UOg53fz7A41+cl8JfHj9kN07dKlieJyRqDqFu4YqdE/arURx14IE8NGi9S6egTn4siOIFf IrTXXWNHmeGjhfb2IXKU5fxMwZX/gc5FEo7k28YKL0pfolExid8mfLsWCguOo/goITuDGEUp Hye8u2P4VzlDqHbU/62J4OOFjrzGv5uycMRbXUadQhIzGlOF5JoUg1alSY5T6vep01I0qcrd +7VOFP4fy7HQ6etosHV1A+I5pBgndbjnqmW0yqBP0zYggSMVcmmVa45aJk1SpR0Rdft36A4l i/oGFMlRiinSNVvEnTJ+j+qguE8UD4i6/5bgIqamI/zUdGvDZqNyYtSRtb7iHwni+HX2EV9s 0F2jc5giQ0MrN80ernXl7B1mXy2YMC7AQGGJtyOk3POevPwV7VrcPf9OUbmk9WgoLlHuGVH+ zJrV2ZyUmBNvt6Z6J22fG7QvG3g/7WL0ItNbJnHLQy7qmWKpNt4+Tbpx/kCCYVXCp70KSq9W xcaQOr3qD0hkd0M7AwAA X-CFilter-Loop: Reflected Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" dept needs to notice every entrance from user to kernel mode to treat every kernel context independently when tracking wait-event dependencies. Roughly, system call and user oriented fault are the cases. Make dept aware of the entrances of x86_64 and add support CONFIG_ARCH_HAS_DEPT_SUPPORT to x86_64. Signed-off-by: Byungchul Park --- arch/x86/Kconfig | 1 + arch/x86/entry/syscall_64.c | 7 +++++++ arch/x86/mm/fault.c | 7 +++++++ 3 files changed, 15 insertions(+) diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig index 5873c9e39919..4b6d9e59d96c 100644 --- a/arch/x86/Kconfig +++ b/arch/x86/Kconfig @@ -38,6 +38,7 @@ config X86_64 select ARCH_HAS_ELFCORE_COMPAT select ZONE_DMA32 select EXECMEM if DYNAMIC_FTRACE + select ARCH_HAS_DEPT_SUPPORT =20 config FORCE_DYNAMIC_FTRACE def_bool y diff --git a/arch/x86/entry/syscall_64.c b/arch/x86/entry/syscall_64.c index b6e68ea98b83..66bd5af5aff1 100644 --- a/arch/x86/entry/syscall_64.c +++ b/arch/x86/entry/syscall_64.c @@ -8,6 +8,7 @@ #include #include #include +#include =20 #define __SYSCALL(nr, sym) extern long __x64_##sym(const struct pt_regs *); #define __SYSCALL_NORETURN(nr, sym) extern long __noreturn __x64_##sym(con= st struct pt_regs *); @@ -86,6 +87,12 @@ static __always_inline bool do_syscall_x32(struct pt_reg= s *regs, int nr) /* Returns true to return using SYSRET, or false to use IRET */ __visible noinstr bool do_syscall_64(struct pt_regs *regs, int nr) { + /* + * This is a system call from user mode. Make dept work with a + * new kernel mode context. + */ + dept_update_cxt(); + add_random_kstack_offset(); nr =3D syscall_enter_from_user_mode(regs, nr); =20 diff --git a/arch/x86/mm/fault.c b/arch/x86/mm/fault.c index 296d294142c8..241537ce47fe 100644 --- a/arch/x86/mm/fault.c +++ b/arch/x86/mm/fault.c @@ -20,6 +20,7 @@ #include #include /* find_and_lock_vma() */ #include +#include =20 #include /* boot_cpu_has, ... */ #include /* dotraplinkage, ... */ @@ -1220,6 +1221,12 @@ void do_user_addr_fault(struct pt_regs *regs, tsk =3D current; mm =3D tsk->mm; =20 + /* + * This fault comes from user mode. Make dept work with a new + * kernel mode context. + */ + dept_update_cxt(); + if (unlikely((error_code & (X86_PF_USER | X86_PF_INSTR)) =3D=3D X86_PF_IN= STR)) { /* * Whoops, this is kernel mode code trying to execute from --=20 2.17.1 From nobody Fri Dec 19 15:51:27 2025 Received: from invmail4.hynix.com (exvmail4.hynix.com [166.125.252.92]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 2E72B266B76; Mon, 19 May 2025 09:18:44 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=166.125.252.92 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747646330; cv=none; b=fQxmiiate4GSyaFSl6bW5P5ZEF9PQ7pHwkx6xBrYrxMaryDSa24cQjUgHKJI/5mBewatE4bgk6h3LtqY6osTQmdsnN1RNl3JlpHjp3ib5hgGaW/X+yWKADaQ+HEbRHnmSCu5Z0YiBrilQBahoNhNhEaewUOZ/FkQNYXWiSAKpro= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747646330; c=relaxed/simple; bh=BG7AZBSjSmF6NrOFDpw8Rk4CMPwauNC98FDvthYcPSo=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References; b=TcKULzPIS3gyvRzWIn6Joe44xPe4s/lIYMZvmdEOR/BFGK4NziKYsCNSMXEVXlcl7vWqGRxHlrhpNAo8FTzwyUt8OtDEnZxV5vbGlQGnxdquG1q1HY90WVFBdAFUo6t8etOLuk/Ja86RpN77hHOsaH6LNSXA8AjbmMm/XLC575c= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com; spf=pass smtp.mailfrom=sk.com; arc=none smtp.client-ip=166.125.252.92 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=sk.com X-AuditID: a67dfc5b-669ff7000002311f-74-682af76de41b From: Byungchul Park To: linux-kernel@vger.kernel.org Cc: kernel_team@skhynix.com, torvalds@linux-foundation.org, damien.lemoal@opensource.wdc.com, linux-ide@vger.kernel.org, adilger.kernel@dilger.ca, linux-ext4@vger.kernel.org, mingo@redhat.com, peterz@infradead.org, will@kernel.org, tglx@linutronix.de, rostedt@goodmis.org, joel@joelfernandes.org, sashal@kernel.org, daniel.vetter@ffwll.ch, duyuyang@gmail.com, johannes.berg@intel.com, tj@kernel.org, tytso@mit.edu, willy@infradead.org, david@fromorbit.com, amir73il@gmail.com, gregkh@linuxfoundation.org, kernel-team@lge.com, linux-mm@kvack.org, akpm@linux-foundation.org, mhocko@kernel.org, minchan@kernel.org, hannes@cmpxchg.org, vdavydov.dev@gmail.com, sj@kernel.org, jglisse@redhat.com, dennis@kernel.org, cl@linux.com, penberg@kernel.org, rientjes@google.com, vbabka@suse.cz, ngupta@vflare.org, linux-block@vger.kernel.org, josef@toxicpanda.com, linux-fsdevel@vger.kernel.org, jack@suse.cz, jlayton@kernel.org, dan.j.williams@intel.com, hch@infradead.org, djwong@kernel.org, dri-devel@lists.freedesktop.org, rodrigosiqueiramelo@gmail.com, melissa.srw@gmail.com, hamohammed.sa@gmail.com, harry.yoo@oracle.com, chris.p.wilson@intel.com, gwan-gyeong.mun@intel.com, max.byungchul.park@gmail.com, boqun.feng@gmail.com, longman@redhat.com, yskelg@gmail.com, yunseong.kim@ericsson.com, yeoreum.yun@arm.com, netdev@vger.kernel.org, matthew.brost@intel.com, her0gyugyu@gmail.com Subject: [PATCH v16 09/42] arm64, dept: add support CONFIG_ARCH_HAS_DEPT_SUPPORT to arm64 Date: Mon, 19 May 2025 18:17:53 +0900 Message-Id: <20250519091826.19752-10-byungchul@sk.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20250519091826.19752-1-byungchul@sk.com> References: <20250519091826.19752-1-byungchul@sk.com> X-Brightmail-Tracker: H4sIAAAAAAAAAzWSa0hTYRjHe99zm6vVaUWd7jEYlWJldHmCCulDnT4UQRRlQa48tdVcNcuc FGjNMs1RkkkXdVpM0Vl2FnSdLcWVibXS1MRLjtKNNg1zM8suW9KXhx/P5ff/8kgI+UtqpkSj Oy7odSqtgpaSUv+EkmjdcKR6aYNDAYGhTBJu3rXS4LpTgcB6Px2Dt24jtAZ9CH42viEgP8+F oLink4D7zi4E9rIzNDR9mgjNgQEa6vOyaTh76y4Nb7+MYui4mouhQtwM3ZZeEhoulWDI99Jw I/8sDhUPhhFLOQOWNCW4y64zMNoTA/VdLRTY26PgWmEHDU/t9SQ4H7oxND2+SUOX9Q8FDc6X JARNs8B1OYeCyv4SGr4ELQRYAgMMvHOYMTjN06DKGBKe+/abghc5Dgznbt/D0PzhCYLqzI8Y RGsLDbUBHwabmEfAj9I6BG6Tn4GMiyMM3Eg3IcjOuEqCsWMF/PweSi4YioH0oioSKn+1oNi1 vLXQivha3wDBG20n+R+B9zRvD5pJ/lUJxz+63snwxup2hjeLJ3hbWSR/66kX88WDAYoXyy/Q vDiYy/BZ/mbM979+zWydHSddkyBoNcmCfsm6eKn6xVAlPlovT+mrzsVpyDQpC0VIOHY51/3K hf9zf2ktCjPNLuDa2kaIME9l53O2nF4qC0klBNsynmst+PBvaQq7i3v3+A0dZpJVck/Si6kw y9iVXMblVnpMOo+rqHL8E0WE+u3ZYwFydgXXXFFIhqUcmx/B/cnIJMcOZnDPy9rIS0hmRuPK kVyjS05UabTLF6sNOk3K4v1HEkUU+i/L6dHdD9Gga1sNYiVIMUFWZV+kllOq5CRDYg3iJIRi qqzctlAtlyWoDKmC/she/QmtkFSDZklIxXTZsuDJBDl7UHVcOCwIRwX9/ymWRMxMQ9xB0+Sv 95SrlJ4D81P87tXvlVv6ivI+dis7BU98lDf1lCFt+0pxntObuqmRcG94tqNx55XJcWtik6eV 9hi0bFPc7wPnP3Xv+fYguM+TQO7uOzQULfeZPJnDpmNR43sXrb9ijl0dP1dTLQ7n+pi14y4c c6ibRg6tT2k8X/DZuNU/R0EmqVUxkYQ+SfUXE5tQ7VsDAAA= X-Brightmail-Tracker: H4sIAAAAAAAAAzWSf0iTeRzH+36fn65WDzuph6ROBmYZdQkZHyiiCOqh644oq7v6w3b1cBtO 022ZBoFrGp3LkR4mZ802jSnzUdcmnJUTUbKbkpouNTFTC038sX64lWl2s+ifDy8+7w+v9z8f llA4qbWsJtUg6lJVWiUtI2W/7jRtSfkQp94WGt8IwdmrJNyqlWjoqqlCINUZMUw8PAB9oSkE 8487CSgu6kJgH3lOQF3rEAJv5WUael6tBH8wQIOvyEyDqbyWhieTCxgGbxRiqHL/Ai8cYyS0 Xy/DUDxBw81iEw6P1xjmHE4GHNkxMFpZwsDCSDz4hnopaLH6KPAObIZ/SgdpaPD6SGitH8XQ c/8WDUPSFwraW/8jIWSJgq6CfAqqZ8pomAw5CHAEAwx0N9kwtNpWgysnbL3yfpGCR/lNGK7c uYvB/+wBgsarwxjcUi8NLcEpDB53EQGfKh4iGLVMM5B7bY6Bm0YLAnPuDRJyBhNg/mO42Tob D8bbLhKqP/eiPbsFqVRCQstUgBByPBeET8GntOAN2UihrYwX7pU8Z4ScxgFGsLnPC57KOKG8 YQIL9ndBSnA7/6IF97tCRsib9mNhpqODObzupGzXWVGryRB1P+0+LVM/mq3GaT5F5nhjIc5G llV5KILlue38TEULWmKai+X7++eIJY7konlP/hiVh2QswfUu5/usz74e/cD9znff76SXmORi +AdGO7XEcm4Hn1vQR3+T/shXuZq+iiLC+wHztwIFl8D7q0rJ60hmQ8ucKFKTmpGi0mgTtuqT 1VmpmsytZ86luFH4gxyXFgrq0WzPgWbEsUi5Qu7yblIrKFWGPiulGfEsoYyUOz0b1Qr5WVXW RVF3Lkl3Xivqm1EUSyrXyA+eEE8ruD9VBjFZFNNE3fcUsxFrs9F+uXWffa7NerTkzPGxvYvR wZg6nb/ij3TbISkT/Xbq7+WSf8O6hrEeMzOyfSf8/FIqX/9vbIf62GT0nTitPKo5sTjKdW/V mkIImWv2WIZNSdMT7fVGU9bRw/NvDNWBI9pkw0KdoXv00nEI2RMD6W9JNj12uKZ+3LmonP6o PKQk9WpVfByh06v+B0+8G4g9AwAA X-CFilter-Loop: Reflected Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" dept needs to notice every entrance from user to kernel mode to treat every kernel context independently when tracking wait-event dependencies. Roughly, system call and user oriented fault are the cases. Make dept aware of the entrances of arm64 and add support CONFIG_ARCH_HAS_DEPT_SUPPORT to arm64. Signed-off-by: Byungchul Park --- arch/arm64/Kconfig | 1 + arch/arm64/kernel/syscall.c | 7 +++++++ arch/arm64/mm/fault.c | 7 +++++++ 3 files changed, 15 insertions(+) diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig index a182295e6f08..6c69598a6423 100644 --- a/arch/arm64/Kconfig +++ b/arch/arm64/Kconfig @@ -279,6 +279,7 @@ config ARM64 select HAVE_SOFTIRQ_ON_OWN_STACK select USER_STACKTRACE_SUPPORT select VDSO_GETRANDOM + select ARCH_HAS_DEPT_SUPPORT help ARM 64-bit (AArch64) Linux support. =20 diff --git a/arch/arm64/kernel/syscall.c b/arch/arm64/kernel/syscall.c index c442fcec6b9e..bbd306335179 100644 --- a/arch/arm64/kernel/syscall.c +++ b/arch/arm64/kernel/syscall.c @@ -7,6 +7,7 @@ #include #include #include +#include =20 #include #include @@ -96,6 +97,12 @@ static void el0_svc_common(struct pt_regs *regs, int scn= o, int sc_nr, * (Similarly for HVC and SMC elsewhere.) */ =20 + /* + * This is a system call from user mode. Make dept work with a + * new kernel mode context. + */ + dept_update_cxt(); + if (flags & _TIF_MTE_ASYNC_FAULT) { /* * Process the asynchronous tag check fault before the actual diff --git a/arch/arm64/mm/fault.c b/arch/arm64/mm/fault.c index ec0a337891dd..0fcc3dc9c2a9 100644 --- a/arch/arm64/mm/fault.c +++ b/arch/arm64/mm/fault.c @@ -26,6 +26,7 @@ #include #include #include +#include =20 #include #include @@ -616,6 +617,12 @@ static int __kprobes do_page_fault(unsigned long far, = unsigned long esr, if (!(mm_flags & FAULT_FLAG_USER)) goto lock_mmap; =20 + /* + * This fault comes from user mode. Make dept work with a new + * kernel mode context. + */ + dept_update_cxt(); + vma =3D lock_vma_under_rcu(mm, addr); if (!vma) goto lock_mmap; --=20 2.17.1 From nobody Fri Dec 19 15:51:27 2025 Received: from invmail4.hynix.com (exvmail4.hynix.com [166.125.252.92]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 32C8826B96B; Mon, 19 May 2025 09:18:44 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=166.125.252.92 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747646329; cv=none; b=ZS2eNHcjNaFfWOm9poMSOy+31iB+u5N76LQB5Cy/T1DBZnQOgKjR9sWJuKyyuaAygisruyIfhxE1ld9oFJzDg71WlCJ2rRfQUTnJRj/Xh30T2QCX6e4mU/ZwtlyWGY9jBydM9fSWsUDLTfX+HrS2arVuNlhh3oGNASYJu+kmeaY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747646329; c=relaxed/simple; bh=KlggbR8P/n36oRZlJZsBth/VG91g9Dz5bdB/P0jqAy0=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References; b=I5tSieYR1ckCwfyCrZmtOIRaY0ZyFvyida6pHMN9J2AQWcxsQ8274ay8lCf5IJebYIohvPhlSuYpHjOe3RYHNnx6W8rrj/Ujtgw2u4kPmpmcRpuJ0t8aydzIger6YmVWpIGjRC/SU9HT8Sx1pQBzzrq2su+FYP6cyP7+Pmk7KAQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com; spf=pass smtp.mailfrom=sk.com; arc=none smtp.client-ip=166.125.252.92 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=sk.com X-AuditID: a67dfc5b-669ff7000002311f-75-682af76d6e83 From: Byungchul Park To: linux-kernel@vger.kernel.org Cc: kernel_team@skhynix.com, torvalds@linux-foundation.org, damien.lemoal@opensource.wdc.com, linux-ide@vger.kernel.org, adilger.kernel@dilger.ca, linux-ext4@vger.kernel.org, mingo@redhat.com, peterz@infradead.org, will@kernel.org, tglx@linutronix.de, rostedt@goodmis.org, joel@joelfernandes.org, sashal@kernel.org, daniel.vetter@ffwll.ch, duyuyang@gmail.com, johannes.berg@intel.com, tj@kernel.org, tytso@mit.edu, willy@infradead.org, david@fromorbit.com, amir73il@gmail.com, gregkh@linuxfoundation.org, kernel-team@lge.com, linux-mm@kvack.org, akpm@linux-foundation.org, mhocko@kernel.org, minchan@kernel.org, hannes@cmpxchg.org, vdavydov.dev@gmail.com, sj@kernel.org, jglisse@redhat.com, dennis@kernel.org, cl@linux.com, penberg@kernel.org, rientjes@google.com, vbabka@suse.cz, ngupta@vflare.org, linux-block@vger.kernel.org, josef@toxicpanda.com, linux-fsdevel@vger.kernel.org, jack@suse.cz, jlayton@kernel.org, dan.j.williams@intel.com, hch@infradead.org, djwong@kernel.org, dri-devel@lists.freedesktop.org, rodrigosiqueiramelo@gmail.com, melissa.srw@gmail.com, hamohammed.sa@gmail.com, harry.yoo@oracle.com, chris.p.wilson@intel.com, gwan-gyeong.mun@intel.com, max.byungchul.park@gmail.com, boqun.feng@gmail.com, longman@redhat.com, yskelg@gmail.com, yunseong.kim@ericsson.com, yeoreum.yun@arm.com, netdev@vger.kernel.org, matthew.brost@intel.com, her0gyugyu@gmail.com Subject: [PATCH v16 10/42] dept: distinguish each work from another Date: Mon, 19 May 2025 18:17:54 +0900 Message-Id: <20250519091826.19752-11-byungchul@sk.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20250519091826.19752-1-byungchul@sk.com> References: <20250519091826.19752-1-byungchul@sk.com> X-Brightmail-Tracker: H4sIAAAAAAAAAzWSbUxTZxTHfZ57e++lo8tNp3h9SZxNnIqZjkX0ZHNTtw88WWKyzG9oInVc bV1pTYsFNIsgYBxdq+KA+EJpkXRNW7W2fEClBEvoBKai1oqkghBlIwJdwJYXca7F7MvJL+ef /+98ORwlD0mWc2ptoajXKjUKRkpLx9Ptn2qnM1WfNcRWQ/z1KRouXfMw0HvVjcDTXIZhtDMH niTGELy5e5+CuppeBPahZxQ0hwYQBJwnGHj04kMIx2MMdNWYGCi/fI2BB6/mMURrqzG4fbtg 0DFCQ8+ZRgx1owxcrCvHyfE3hlmHiwVH6RoYdl5gYX4oC7oGIhII9G+A89YoA62BLhpCLcMY Ht28xMCA550EekJ3aEhYVkDvWbMErkw0MvAq4aDAEY+x8LDdhiFkywBvRVJ4cupfCfxhbsdw suk6hvDTWwjaTj3H4PNEGOiIj2Hw+2oomPu9E8GwZZyFyl9nWbhYZkFgqqyloSKaDW9mkpfr X2dBWYOXhitvI2jHV8Rj9SDSMRajSIW/iMzFHzMkkLDRpLtRIDcuPGNJRVs/S2y+I8TvzCSX W0cxsU/GJcTn+oUhvslqllSNhzGZuHeP/X5lrnRbvqhRG0X9pq/zpCpvpfHwBFvcFPwHl6JW pgpxnMBvFqJDmiqUtoDjjZ1sihl+rdDXN0uleDH/seA3j0iqkJSj+MgHwpP6pygVfMR/K9jt A0yKaX6NMDM1j1Ms47cIdlMUv5euEtze9gVRWnLfb+pY6Mr5bCHsttIpqcCfSxNqO6vZ94Vl wm1nH30GyWxokQvJ1VpjgVKt2bxRVaJVF2/8UVfgQ8nvcvw8v6cFTfbuDiKeQ4p0mTewXiWX KI2GkoIgEjhKsVjm8q9TyWX5ypKjol63T39EIxqCaAVHK5bKPk8U5cv5g8pC8SdRPCzq/08x l7a8FOVucvE7bwxayxfpCu8/jGXkvQyfzevObcvuOt+8/e3Ui4jzXeGh1ekj25eajzaTrXMt 3obMLwq2dRiXtX3jPm7ZX/9l6dwJVe4nQ6jowW+GCYvP7yhe+ed3uuAGy4G/ehR7Z1YN/mAx Wc/tMmdkT7+Ude/OmdYda8o6ffqxbMna4zdzFLRBpczKpPQG5X+H3qqAWQMAAA== X-Brightmail-Tracker: H4sIAAAAAAAAAzWSa0iTYRTHe5736mrxtoa9LLQYdDOthlqHLiuQ8qXoQgRBfchhL22oKzZb GQSaTkxT0lhiNdumLNOVayvo4kIULYvK0sxkWVpEkjdqW5qaTaMvhx/nz/93vhyWkN2kFKxO nyka9Jp0JS0hJXs25cZl/IrRrvucr4JgoICEa/UuGtpv1yFw3c3BMNCSDO9CgwgmXrwioNzS jsDe94GAu629CHw152jo+DIfOoMjNLRZimjIraqn4fX3SQz+y2UY6jy74aPzKwnPLzowlA/Q cLU8F4fHNwzjzloGnNnLoL/mCgOTfSpo6+2ioNnaRoGvZzVUVPppaPC1kdB6vx9Dx8NrNPS6 pil43vqUhFDJYmgvLabg1rCDhu8hJwHO4AgDbxptGFptkeDOC1vzf/6h4ElxI4b86jsYOt8/ QvC44BMGj6uLhubgIAavx0LA7xstCPpLhhgwXxhn4GpOCYIi82US8vyJMDEWvmwNqCDnupuE W1NdaJtacFW6kNA8OEIIed5Twu/gW1rwhWyk8MzBCw+ufGCEvMc9jGDznBS8NTFCVcMAFuw/ gpTgqT1PC54fZYxQONSJheGXL5l9UYckm4+K6TqTaFirTpFo3WbTiWHmdHXTKM5GDXQhimB5 LoEfcrQwM0xzK/ju7nFihuXcUt5b/JUqRBKW4Lrm8u+s79FMsJBL4u323tkyyS3jx35O4hmW cut5e5Ef/5Mu4evcjbOiiPC+p6h5tivjEvnOukryIpLY0JxaJNfpTRkaXXriGmOaNkuvO70m 9XiGB4UfyHl2svQ+CnQkNyGORcp5UrdvlVZGaUzGrIwmxLOEUi6t9a7UyqRHNVlnRMPxI4aT 6aKxCS1mSeUi6c6DYoqMO6bJFNNE8YRo+J9iNkKRjTZGsQmSWAthUavjI5fGS6Md8sPdAVVa 0l65PzW+QV1dsiMy2Xlpw3Tm9sbEJHNa6oFzCkdyQkHsvfqKwIKnow+iuTLblrVR5oqxqY4p hfVs5cR03KhJW7r1T3Zo/mp7cPe8FGvs+M1d+rnL90e3TO3IH+3ri407syH3m7xwpyJaSRq1 GlUMYTBq/gJ+wP42PAMAAA== X-CFilter-Loop: Reflected Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Workqueue already provides concurrency control. By that, any wait in a work doesn't prevents events in other works with the control enabled. Thus, each work would better be considered a different context. So let dept assign a different context id to each work. Signed-off-by: Byungchul Park --- kernel/workqueue.c | 3 +++ 1 file changed, 3 insertions(+) diff --git a/kernel/workqueue.c b/kernel/workqueue.c index cf6203282737..5a6ab354c416 100644 --- a/kernel/workqueue.c +++ b/kernel/workqueue.c @@ -55,6 +55,7 @@ #include #include #include +#include =20 #include "workqueue_internal.h" =20 @@ -3155,6 +3156,8 @@ __acquires(&pool->lock) =20 lockdep_copy_map(&lockdep_map, &work->lockdep_map); #endif + dept_update_cxt(); + /* ensure we're on the correct CPU */ WARN_ON_ONCE(!(pool->flags & POOL_DISASSOCIATED) && raw_smp_processor_id() !=3D pool->cpu); --=20 2.17.1 From nobody Fri Dec 19 15:51:27 2025 Received: from invmail4.hynix.com (exvmail4.hynix.com [166.125.252.92]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 2E78C267391; Mon, 19 May 2025 09:18:44 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=166.125.252.92 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747646330; cv=none; b=O7cn6uw873mY388ToW3ksPsMNs+/YfVDJyNBkcHJckw9SYxsoY1a2/arlFdvqrLegEaDz07NGekelESBG888Ijn2hoPZE2BwLEPrXhOcz9IJljCmeGDH7GONhYER2h5nient9rZHp8CzZYdpSgqLly/HopoZ8MfGgfkG9qO/H40= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747646330; c=relaxed/simple; bh=XPhndR0BcCicw1+tO0GozjV4Ce2hwqOtkLVx0kL8UdM=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References; b=WWoBVR8FVKe0lxOH5iWxGg8Xh2bCngwsj4Igo9oChImMnqAW77mP/vAdqMC/1ko8HIPl3cv/38DLc61HNof02TsdHk1rEkz/G5kOMXAgLtX9AbJdkt8hXd4Od1u93PxCqAiBYiQqee8Pv3PKeU9PMc+4rFLmxibps9JHN03lj/0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com; spf=pass smtp.mailfrom=sk.com; arc=none smtp.client-ip=166.125.252.92 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=sk.com X-AuditID: a67dfc5b-669ff7000002311f-76-682af76de6e3 From: Byungchul Park To: linux-kernel@vger.kernel.org Cc: kernel_team@skhynix.com, torvalds@linux-foundation.org, damien.lemoal@opensource.wdc.com, linux-ide@vger.kernel.org, adilger.kernel@dilger.ca, linux-ext4@vger.kernel.org, mingo@redhat.com, peterz@infradead.org, will@kernel.org, tglx@linutronix.de, rostedt@goodmis.org, joel@joelfernandes.org, sashal@kernel.org, daniel.vetter@ffwll.ch, duyuyang@gmail.com, johannes.berg@intel.com, tj@kernel.org, tytso@mit.edu, willy@infradead.org, david@fromorbit.com, amir73il@gmail.com, gregkh@linuxfoundation.org, kernel-team@lge.com, linux-mm@kvack.org, akpm@linux-foundation.org, mhocko@kernel.org, minchan@kernel.org, hannes@cmpxchg.org, vdavydov.dev@gmail.com, sj@kernel.org, jglisse@redhat.com, dennis@kernel.org, cl@linux.com, penberg@kernel.org, rientjes@google.com, vbabka@suse.cz, ngupta@vflare.org, linux-block@vger.kernel.org, josef@toxicpanda.com, linux-fsdevel@vger.kernel.org, jack@suse.cz, jlayton@kernel.org, dan.j.williams@intel.com, hch@infradead.org, djwong@kernel.org, dri-devel@lists.freedesktop.org, rodrigosiqueiramelo@gmail.com, melissa.srw@gmail.com, hamohammed.sa@gmail.com, harry.yoo@oracle.com, chris.p.wilson@intel.com, gwan-gyeong.mun@intel.com, max.byungchul.park@gmail.com, boqun.feng@gmail.com, longman@redhat.com, yskelg@gmail.com, yunseong.kim@ericsson.com, yeoreum.yun@arm.com, netdev@vger.kernel.org, matthew.brost@intel.com, her0gyugyu@gmail.com Subject: [PATCH v16 11/42] dept: add a mechanism to refill the internal memory pools on running out Date: Mon, 19 May 2025 18:17:55 +0900 Message-Id: <20250519091826.19752-12-byungchul@sk.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20250519091826.19752-1-byungchul@sk.com> References: <20250519091826.19752-1-byungchul@sk.com> X-Brightmail-Tracker: H4sIAAAAAAAAAzWSfUxTZxjF97739t7baslNRb0iRlNimDgRROezZH4sWeIlC3HJYqJ2yWzk jjbSYoqAuJjA+IgDIU4HKNDaohZoC2JLNrZR0pUVQbOuQi1ISlVijFU+ErSdKOoKm/88+eWc nPP8cxhCdlOUwKi1JwSdVpkrpySkZHp5yxbtPymqtOe9SRB5cYaE5us2CnydVgS27lIMYc8+ GI1OIXj9198ENNT5EJgeThDQPRBC4Gz7noKRR3Hgj8xSMFRXTUHZlesU3Hm2gCFYfx6D1Z4F 982PSbh9rgVDQ5iCpoYyHDtPMMybLTSYSzbCZFsjDQsP02EoFBCBc3wzXDIEKeh1DpEw0DOJ YeS3ZgpCtnciuD0wSEK0di34fqwRQcdMCwXPomYCzJFZGoZdRgwDxlXQVR4rrHz+VgQ3a1wY Kq/ewOC/9zuCvjMPMNhtAQr6I1MYHPY6Al61ehBM1k7TUHF2noam0loE1RX1JJQHd8Drl7HP +hfpUHq5i4SONwG0dxdvM9gQ3z81S/DljiL+VeQuxTujRpK/1cLxvzZO0Hx53zjNG+0FvKMt hb/SG8a8aS4i4u2WHyjePnee5qum/Zif8XrpLxMPSz7NFnLVhYJu6+4jEpXbO0ge9+49GXqg RyWoM6MKiRmO3c49NfYR7/nWjAcvMsUmc2Nj80t6PLuBc9Q8FlUhCUOwgWXcqP4eWjRWsEe5 EnNHjBmGZDdyd8zUoixlP+buD9X837mes3a5llgc08er+5eiMnYH57cayMVOjr0g5obfBPB/ gTXcH21j5DkkNaIPLEim1hZqlOrc7amqYq36ZOrRPI0dxeZlPr2g6EFzvq/ciGWQfLm0y7lJ JRMpC/OLNW7EMYQ8XmpxfKiSSbOVxacEXd43uoJcId+N1jKkfLV0W7QoW8bmKE8IxwThuKB7 72JGnFCCrskvfPdJXMbnZd9m5UxWTBzK21//RP+nIjEl0/325c4DSe2Kn0dCeo83IfjFFv9o 6rDLFGSUP+UUfRZut/a4statmDcVrLF2r1RXJuc0ZSRes2kEjaIy/qAlbXzD5neOq8eWFRzZ ldm6bdUlU9YvvtX7JrovDu7xJIW/TjOwYsVHGXIyX6VMTyF0+cp/ASnhV1paAwAA X-Brightmail-Tracker: H4sIAAAAAAAAAzWSfUzMcRzHfX/PHWe/ncZvmrEzT6GE+HhY6z/fMcbMbEw59Zs7XYe7StlY KU/lIiRKdcJPrivnLpO41kpxpRPlJOlpmEiNukh5uJh/3nvt/d7n/f7nw5GKm/RUTqOLEfU6 lVbJyCjZhlXJC6O/+asXlbcsB8/gCQou37Iw0FhShMBSmkRAT80aeDnUi2Ck4SkJWZmNCK50 vSGhtLYdgaPwCANNbydCs6efAWdmGgPJV28x8OzTKAFtF84SUGRbDx3SewrqzxQQkNXDQE5W MuGVDwQMS2YWpMRZ0F2YzcJoVxA42900VOc6aXC0zodLeW0MPHA4Kagt6yagqfwyA+2W3zTU 1z6mYCjdDxozjDQU9xUw8GlIIkHy9LPwvNJEQK1pMlhTvK3HBn7R8MhYScCxa7cJaH51H0HF iU4CbBY3A9WeXgLstkwSftyoQdCd/pmFo6eGWchJSkeQdvQCBSltwTDy3bucOxgESflWCop/ ulFoCLbkWRCu7u0ncYr9AP7hecFgx5CJwnUFAr6X/YbFKRWtLDbZYrG90B9ffdBD4CtfPTS2 mU8y2Pb1LItTPzcTuM/lYjdO2yZbHSlqNXGiPjBkp0xd5XpM7XOFxrd35qJEVLIkFflwAr9U qOurIcaY4ecILS3D5Bj78jMEu/E9nYpkHMm7xwsvc1+hsWASHyEkSsVe5jiKnyU8k5gxW84v EzqcRvJf53ShyFr5l328fmta9d9TBR8sNBflUWeQzITGmZGvRhcXrdJogwMMUeoEnSY+IGJv tA15H0g6NJpRhgab1lQhnkPKCXKrY55aQaviDAnRVUjgSKWv3Gyfq1bII1UJB0X93nB9rFY0 VCE/jlJOka/dKu5U8LtVMWKUKO4T9f9TgvOZmojKT7OT9icfzn9tOl1/pOLSFutMV9imjlZd wvnAd/ULP0as8IvNqeudwS2KU67bLu26G2a+HaiYd/x36cMuXeDTxTtWXrwTUBeTUVo2efPA Oc3I/HHhIU2d4+O/vN1uzC7R/pI15q/aEhnv27BiwciHpVHuBuee6za/2aKkvxvqwANPrErK oFYF+ZN6g+oPsQjb4DwDAAA= X-CFilter-Loop: Reflected Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" dept engine works in a constrained environment. For example, dept cannot make use of dynamic allocation e.g. kmalloc(). So dept has been using static pools to keep memory chunks dept uses. However, dept would barely work once any of the pools gets run out. So implemented a mechanism for the refill on the lack, using irq work and workqueue that fits on the contrained environment. Signed-off-by: Byungchul Park --- kernel/dependency/dept.c | 108 +++++++++++++++++++++++++----- kernel/dependency/dept_internal.h | 19 ++++-- kernel/dependency/dept_object.h | 10 +-- kernel/dependency/dept_proc.c | 8 +-- 4 files changed, 116 insertions(+), 29 deletions(-) diff --git a/kernel/dependency/dept.c b/kernel/dependency/dept.c index b4a39c81bbc1..ad443e063fdc 100644 --- a/kernel/dependency/dept.c +++ b/kernel/dependency/dept.c @@ -75,6 +75,9 @@ #include #include #include +#include +#include +#include #include "dept_internal.h" =20 static int dept_stop; @@ -143,9 +146,11 @@ static inline struct dept_task *dept_task(void) } \ }) =20 -#define DEPT_INFO_ONCE(s...) pr_warn_once("DEPT_INFO_ONCE: " s) +#define DEPT_INFO_ONCE(s...) pr_warn_once("DEPT_INFO_ONCE: " s) +#define DEPT_INFO(s...) pr_warn("DEPT_INFO: " s) =20 static arch_spinlock_t dept_spin =3D (arch_spinlock_t)__ARCH_SPIN_LOCK_UNL= OCKED; +static arch_spinlock_t dept_pool_spin =3D (arch_spinlock_t)__ARCH_SPIN_LOC= K_UNLOCKED; =20 /* * DEPT internal engine should be cautious in using outside functions @@ -268,6 +273,7 @@ static bool valid_key(struct dept_key *k) =20 #define OBJECT(id, nr) \ static struct dept_##id spool_##id[nr]; \ +static struct dept_##id rpool_##id[nr]; \ static DEFINE_PER_CPU(struct llist_head, lpool_##id); #include "dept_object.h" #undef OBJECT @@ -276,14 +282,74 @@ struct dept_pool dept_pool[OBJECT_NR] =3D { #define OBJECT(id, nr) { \ .name =3D #id, \ .obj_sz =3D sizeof(struct dept_##id), \ - .obj_nr =3D ATOMIC_INIT(nr), \ + .obj_nr =3D nr, \ + .tot_nr =3D nr, \ + .acc_sz =3D ATOMIC_INIT(sizeof(spool_##id) + sizeof(rpool_##id)), \ .node_off =3D offsetof(struct dept_##id, pool_node), \ .spool =3D spool_##id, \ + .rpool =3D rpool_##id, \ .lpool =3D &lpool_##id, }, #include "dept_object.h" #undef OBJECT }; =20 +static void dept_wq_work_fn(struct work_struct *work) +{ + int i; + + for (i =3D 0; i < OBJECT_NR; i++) { + struct dept_pool *p =3D dept_pool + i; + int sz =3D p->tot_nr * p->obj_sz; + void *rpool; + bool need; + + local_irq_disable(); + arch_spin_lock(&dept_pool_spin); + need =3D !p->rpool; + arch_spin_unlock(&dept_pool_spin); + local_irq_enable(); + + if (!need) + continue; + + rpool =3D vmalloc(sz); + + if (!rpool) { + DEPT_STOP("Failed to extend internal resources.\n"); + break; + } + + local_irq_disable(); + arch_spin_lock(&dept_pool_spin); + if (!p->rpool) { + p->rpool =3D rpool; + rpool =3D NULL; + atomic_add(sz, &p->acc_sz); + } + arch_spin_unlock(&dept_pool_spin); + local_irq_enable(); + + if (rpool) + vfree(rpool); + else + DEPT_INFO("Dept object(%s) just got refilled successfully.\n", p->name); + } +} + +static DECLARE_WORK(dept_wq_work, dept_wq_work_fn); + +static void dept_irq_work_fn(struct irq_work *w) +{ + schedule_work(&dept_wq_work); +} + +static DEFINE_IRQ_WORK(dept_irq_work, dept_irq_work_fn); + +static void request_rpool_refill(void) +{ + irq_work_queue(&dept_irq_work); +} + /* * Can use llist no matter whether CONFIG_ARCH_HAVE_NMI_SAFE_CMPXCHG is * enabled or not because NMI and other contexts in the same CPU never @@ -319,19 +385,31 @@ static void *from_pool(enum object_t t) /* * Try static pool. */ - if (atomic_read(&p->obj_nr) > 0) { - int idx =3D atomic_dec_return(&p->obj_nr); + arch_spin_lock(&dept_pool_spin); + + if (!p->obj_nr) { + p->spool =3D p->rpool; + p->obj_nr =3D p->rpool ? p->tot_nr : 0; + p->rpool =3D NULL; + request_rpool_refill(); + } + + if (p->obj_nr) { + void *ret; + + p->obj_nr--; + ret =3D p->spool + (p->obj_nr * p->obj_sz); + arch_spin_unlock(&dept_pool_spin); =20 - if (idx >=3D 0) - return p->spool + (idx * p->obj_sz); + return ret; } + arch_spin_unlock(&dept_pool_spin); =20 - DEPT_INFO_ONCE("---------------------------------------------\n" - " Some of Dept internal resources are run out.\n" - " Dept might still work if the resources get freed.\n" - " However, the chances are Dept will suffer from\n" - " the lack from now. Needs to extend the internal\n" - " resource pools. Ask max.byungchul.park@gmail.com\n"); + DEPT_INFO("------------------------------------------\n" + " Dept object(%s) is run out.\n" + " Dept is trying to refill the object.\n" + " Nevertheless, if it fails, Dept will stop.\n", + p->name); return NULL; } =20 @@ -2956,8 +3034,8 @@ void __init dept_init(void) pr_info("... DEPT_MAX_ECXT_HELD : %d\n", DEPT_MAX_ECXT_HELD); pr_info("... DEPT_MAX_SUBCLASSES : %d\n", DEPT_MAX_SUBCLASSES); #define OBJECT(id, nr) \ - pr_info("... memory used by %s: %zu KB\n", \ - #id, B2KB(sizeof(struct dept_##id) * nr)); + pr_info("... memory initially used by %s: %zu KB\n", \ + #id, B2KB(sizeof(spool_##id) + sizeof(rpool_##id))); #include "dept_object.h" #undef OBJECT #define HASH(id, bits) \ @@ -2965,6 +3043,6 @@ void __init dept_init(void) #id, B2KB(sizeof(struct hlist_head) * (1 << (bits)))); #include "dept_hash.h" #undef HASH - pr_info("... total memory used by objects and hashs: %zu KB\n", B2KB(mem_= total)); + pr_info("... total memory initially used by objects and hashs: %zu KB\n",= B2KB(mem_total)); pr_info("... per task memory footprint: %zu bytes\n", sizeof(struct dept_= task)); } diff --git a/kernel/dependency/dept_internal.h b/kernel/dependency/dept_int= ernal.h index 6b39e5a2a830..b2a44632ee4d 100644 --- a/kernel/dependency/dept_internal.h +++ b/kernel/dependency/dept_internal.h @@ -23,9 +23,19 @@ struct dept_pool { size_t obj_sz; =20 /* - * the number of the static array + * the remaining number of the object in spool */ - atomic_t obj_nr; + int obj_nr; + + /* + * the number of the object in spool + */ + int tot_nr; + + /* + * accumulated amount of memory used by the object in byte + */ + atomic_t acc_sz; =20 /* * offset of ->pool_node @@ -35,9 +45,10 @@ struct dept_pool { /* * pointer to the pool */ - void *spool; + void *spool; /* static pool */ + void *rpool; /* reserved pool */ struct llist_head boot_pool; - struct llist_head __percpu *lpool; + struct llist_head __percpu *lpool; /* local pool */ }; =20 enum object_t { diff --git a/kernel/dependency/dept_object.h b/kernel/dependency/dept_objec= t.h index 0b7eb16fe9fb..4f936adfa8ee 100644 --- a/kernel/dependency/dept_object.h +++ b/kernel/dependency/dept_object.h @@ -6,8 +6,8 @@ * nr: # of the object that should be kept in the pool. */ =20 -OBJECT(dep, 1024 * 8) -OBJECT(class, 1024 * 8) -OBJECT(stack, 1024 * 32) -OBJECT(ecxt, 1024 * 16) -OBJECT(wait, 1024 * 32) +OBJECT(dep, 1024 * 4 * 2) +OBJECT(class, 1024 * 4) +OBJECT(stack, 1024 * 4 * 8) +OBJECT(ecxt, 1024 * 4 * 2) +OBJECT(wait, 1024 * 4 * 4) diff --git a/kernel/dependency/dept_proc.c b/kernel/dependency/dept_proc.c index 97beaf397715..f28992834588 100644 --- a/kernel/dependency/dept_proc.c +++ b/kernel/dependency/dept_proc.c @@ -74,12 +74,10 @@ static int dept_stats_show(struct seq_file *m, void *v) { int r; =20 - seq_puts(m, "Availability in the static pools:\n\n"); + seq_puts(m, "Accumulated amount of memory used by pools:\n\n"); #define OBJECT(id, nr) \ - r =3D atomic_read(&dept_pool[OBJECT_##id].obj_nr); \ - if (r < 0) \ - r =3D 0; \ - seq_printf(m, "%s\t%d/%d(%d%%)\n", #id, r, nr, (r * 100) / (nr)); + r =3D atomic_read(&dept_pool[OBJECT_##id].acc_sz); \ + seq_printf(m, "%s\t%d KB\n", #id, r / 1024); #include "dept_object.h" #undef OBJECT =20 --=20 2.17.1 From nobody Fri Dec 19 15:51:27 2025 Received: from invmail4.hynix.com (exvmail4.hynix.com [166.125.252.92]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 2E6BC26657D; Mon, 19 May 2025 09:18:44 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=166.125.252.92 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747646331; cv=none; b=ce0EJtwpYJuXYBQHqIhvJVmDLm0mxuiF0HYAjtWX9VXXfcKvM3S1eTMKwZUqvpRfjyghc2QLwcqtSmP4DVp/HmGWtDVHquK0sfnBnYOAGk/GTBExpFE8MPedSZIVTlMXrO8HBru6CcmU6O4X2ah/l6eb3JGEX83Efr8Qz3/0Qog= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747646331; c=relaxed/simple; bh=7s1NGY+0taVVLGCaGeZpYQO8VebthwKwiBfZuZTjF3g=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References; b=H2XuvlgPgEbOlM9RkUcZcGxG6wGDfVNWdzfVtmRcBDwYPtSH5wpW/yX+XR/vEuP3z1OQuhH+9zktHGhlnf3WrJp8A9sLWZIGGX9UXQjCZYpBivJkzHDvkBh6lzqtibTaVGpPPrH2J+w7DzjiARsQvP54uOn19uBiUovRAOfTCEQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com; spf=pass smtp.mailfrom=sk.com; arc=none smtp.client-ip=166.125.252.92 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=sk.com X-AuditID: a67dfc5b-681ff7000002311f-94-682af76e9ead From: Byungchul Park To: linux-kernel@vger.kernel.org Cc: kernel_team@skhynix.com, torvalds@linux-foundation.org, damien.lemoal@opensource.wdc.com, linux-ide@vger.kernel.org, adilger.kernel@dilger.ca, linux-ext4@vger.kernel.org, mingo@redhat.com, peterz@infradead.org, will@kernel.org, tglx@linutronix.de, rostedt@goodmis.org, joel@joelfernandes.org, sashal@kernel.org, daniel.vetter@ffwll.ch, duyuyang@gmail.com, johannes.berg@intel.com, tj@kernel.org, tytso@mit.edu, willy@infradead.org, david@fromorbit.com, amir73il@gmail.com, gregkh@linuxfoundation.org, kernel-team@lge.com, linux-mm@kvack.org, akpm@linux-foundation.org, mhocko@kernel.org, minchan@kernel.org, hannes@cmpxchg.org, vdavydov.dev@gmail.com, sj@kernel.org, jglisse@redhat.com, dennis@kernel.org, cl@linux.com, penberg@kernel.org, rientjes@google.com, vbabka@suse.cz, ngupta@vflare.org, linux-block@vger.kernel.org, josef@toxicpanda.com, linux-fsdevel@vger.kernel.org, jack@suse.cz, jlayton@kernel.org, dan.j.williams@intel.com, hch@infradead.org, djwong@kernel.org, dri-devel@lists.freedesktop.org, rodrigosiqueiramelo@gmail.com, melissa.srw@gmail.com, hamohammed.sa@gmail.com, harry.yoo@oracle.com, chris.p.wilson@intel.com, gwan-gyeong.mun@intel.com, max.byungchul.park@gmail.com, boqun.feng@gmail.com, longman@redhat.com, yskelg@gmail.com, yunseong.kim@ericsson.com, yeoreum.yun@arm.com, netdev@vger.kernel.org, matthew.brost@intel.com, her0gyugyu@gmail.com Subject: [PATCH v16 12/42] dept: record the latest one out of consecutive waits of the same class Date: Mon, 19 May 2025 18:17:56 +0900 Message-Id: <20250519091826.19752-13-byungchul@sk.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20250519091826.19752-1-byungchul@sk.com> References: <20250519091826.19752-1-byungchul@sk.com> X-Brightmail-Tracker: H4sIAAAAAAAAAzWSbUxTZxTHfZ772krNTWHbHSyZqXMqRicLbmdTlsUvPkt8mVkWI27ZbuTG VqGwVlCWmICik3ZU0RUmUCw4L7Wtii2J3bQGKYJocCgMgQGbuJEhb47ZOpC9lM59OfnlnPP/ nS+Hp7StTCJvMO6RTUYpU8eqafV4XM0K45Nk/arSxkUQfnyEhqoLXhY6znsQeBsKMYxcXw/3 ImMInrZ/T0G5vQNBzf0BChpaBhEEXQdY6PxlAXSFJ1los1tZOHj6Agt3Rmcx9Jcdx+DxbYSf lGEabh2rxVA+wkJl+UEcLb9hmFbcHCgFi2HIVcHB7P0UaBvsZiDYtxxOVvezcCXYRkNLYAhD 53dVLAx6/2HgVssNGiK2JOgoLWHg3EQtC6MRhQIlPMnB3UYnhhbn81BfFBUe/uNvBlpLGjEc /uYihq7eywiuHvkZg8/bzUIoPIbB77NTMFN3HcGQbZyDQ19Oc1BZaENgPVRGQ1H/anj6Z/Sy 43EKFJ6qp+HcX93o3TTirfYiEhqbpEiRfy+ZCf/AkmDESZObtSL5tmKAI0VX+zji9OUSvyuZ nL4ygknNVJghPncxS3xTxzliGe/CZOL2be79l9LVazPkTEOebHrtnU/V+puFE2zOr/P3Wdoc uAA1qyyI50UhVaxyr7UgVQzPTHWiOWaFJWJPzzQ1xwnCQtFfMsxYkJqnhO754j1Hb2wpXpDE R4FRPMe0sFisc/0eY43whtg7aGf+k74seuobYyJVtN9nDcWyWmG12OWppuekolCpEuvOhp4F XhSvuXroY0jjRPPcSGsw5mVJhszUlfp8o2Hfyh3ZWT4UfS9l/+z2AJrq+KAJCTzSxWnqg8v0 WkbKM+dnNSGRp3QJGrd/qV6ryZDyP5dN2Z+YcjNlcxNK4mndC5rXI3sztMJOaY+8W5ZzZNP/ U8yrEgtQ+nROgdknzTzkS3PHHvz4qCwQP7puc25OXOtXCWknh1tfbU74MGNm6NIrbzJvbWh+ zt5uDoR3vbftRFCudG2puGaLJ8oG5evdb/eHbZ60Jw8+2nVpgfVjTXrSQKkju26N1bDuqEO5 XLxp05qt3tq+s+3FiUyq2PBZ8fg27YH0/FD5FzrarJdSkimTWfoXSiC0w1oDAAA= X-Brightmail-Tracker: H4sIAAAAAAAAAzWSa0yTZxTHfZ73VorV10r0VbY4m4BaL4PEmmM2jZ/w2ZI5wxcUb7yZ72wj FNMKAw0LCBqEQdCl4gVYBS2kLYgtS5hSJKDVghc2ugoIKMQZG24OaRnK1ILxy8kv53/O73w5 MkppZZbLdPqjkkEvJqtYOS3f8VXuev2UWhvT7scQmMynoeyanYXOOhsCe0MOBv+d7fA4OILg 7YNHFJSaOhFcHuynoME9gMBVc4KFrucLwBsYZ8FjKmQht+oaC38Oz2DoO3cWg83xHTy1vKCh o6QSQ6mfhUuluThUXmKYtlg5sGRHwVDNRQ5mBmPBM+BjoK3cw4Crdy1cqOhjocnlocHdOISh 60YZCwP29wx0uO/RECyOhM4zRQzUjlWyMBy0UGAJjHPwV4sZg9u8BOrzQtZTr98xcLeoBcOp K9cxeHtuImjOf4bBYfex0BYYweB0mCh4U30HwVDxKAcnf5nm4FJOMYLCk+doyOvTwNv/QpfL J2Mh57d6Gmr/96FtW4m9wo5I28g4RfKcP5E3gb9Z4gqaadJeKZA/LvZzJK+5lyNmRxpx1qhJ VZMfk8sTAYY4rKdZ4pg4y5GCUS8mYw8fcjs/T5R/fVBK1qVLhi+3Jsm17Tlj7JF/wjMKPOU4 G90OK0BhMoHfKFyd6EKzzPKrhO7uaWqWI/gvBGfRC6YAyWUU7wsXHpf3zA0t5kXhVeMwnmWa jxKqa/6dYwW/SegZMDEfpSsEW33LnCgs1O8tbJvbVfIawWuroEuQ3IzmWVGETp+eIuqSNRuM h7WZel3Ghh9SUxwo9EGWrJkzjWiya3sr4mVINV9R71qjVTJiujEzpRUJMkoVobA6V2uVioNi 5jHJkHrAkJYsGVtRpIxWLVV8myAlKflD4lHpsCQdkQyfUiwLW56NqvjwFUkxC33rEjc/9930 H890T+Tf/6abMVmzdL/GNzzxxHk7VtqyprLT5+8e3a9hv/feqt5ML/OfJ4diftTsrV3irfy9 LGJL9C2XxbNI68zw1/Xrd0TeiCsJuqJvi1WJa3xxi2eCjHyQ01urK3JPpFqebNrzc7R6n5CQ GP/ZLrWKNmrFWDVlMIofAJeO0hA9AwAA X-CFilter-Loop: Reflected Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" The current code records all the waits for later use to track relation between waits and events within each context. However, since the same class is handled the same way, it'd be okay to record only one on behalf of the others if they all have the same class. Even though it's the ideal to search the whole history buffer for that, since it'd cost too high, alternatively, let's keep the latest one when the same class'ed waits consecutively appear. Signed-off-by: Byungchul Park --- kernel/dependency/dept.c | 21 ++++++++++++++++++++- 1 file changed, 20 insertions(+), 1 deletion(-) diff --git a/kernel/dependency/dept.c b/kernel/dependency/dept.c index ad443e063fdc..a2384f3148c5 100644 --- a/kernel/dependency/dept.c +++ b/kernel/dependency/dept.c @@ -1486,9 +1486,28 @@ static struct dept_wait_hist *new_hist(void) return wh; } =20 +static struct dept_wait_hist *last_hist(void) +{ + int pos_n =3D hist_pos_next(); + struct dept_wait_hist *wh_n =3D hist(pos_n); + + /* + * This is the first try. + */ + if (!pos_n && !wh_n->wait) + return NULL; + + return hist(pos_n + DEPT_MAX_WAIT_HIST - 1); +} + static void add_hist(struct dept_wait *w, unsigned int wg, unsigned int ct= xt_id) { - struct dept_wait_hist *wh =3D new_hist(); + struct dept_wait_hist *wh; + + wh =3D last_hist(); + + if (!wh || wh->wait->class !=3D w->class || wh->ctxt_id !=3D ctxt_id) + wh =3D new_hist(); =20 if (likely(wh->wait)) put_wait(wh->wait); --=20 2.17.1 From nobody Fri Dec 19 15:51:27 2025 Received: from invmail4.hynix.com (exvmail4.hynix.com [166.125.252.92]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 2249326FA7B; Mon, 19 May 2025 09:18:50 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=166.125.252.92 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747646336; cv=none; b=La7AulW3k6/yzen0dPMAJni3eRVecuSvDR2e5kvJatpLC8HkJbr4KcX36NuM8UCsqLr0m8Tgltq0KwgXGb9J4UL9LeEcbYf8Y81OvgheG8N5ejJGoyXMWezqZQT8RWQrDELNo4IBPceUqOt3eCmqkGTAiSGHQOc28NGccJ2YKLA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747646336; c=relaxed/simple; bh=mVczWw5AJvTGaUa2k+kGuAdITLqzYAqLiO2CNQ+CNGE=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References; b=ORE1fSdhIPwPHMUW0RvcHoz0plL3w2ozVIPcGnPT0I6woXhgGmtUjP3DO3NKRnqTGZ6q9z9E6qdol8hhG4/6K/9sBWCbusB+HYjAXhBi5lNWxzKCZJ0Gi3coyyB12Rr5J/nGE+QEcCkStW4xph608peCGqUGNbqhqd6qVs8DKFQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com; spf=pass smtp.mailfrom=sk.com; arc=none smtp.client-ip=166.125.252.92 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=sk.com X-AuditID: a67dfc5b-681ff7000002311f-a4-682af76e43e1 From: Byungchul Park To: linux-kernel@vger.kernel.org Cc: kernel_team@skhynix.com, torvalds@linux-foundation.org, damien.lemoal@opensource.wdc.com, linux-ide@vger.kernel.org, adilger.kernel@dilger.ca, linux-ext4@vger.kernel.org, mingo@redhat.com, peterz@infradead.org, will@kernel.org, tglx@linutronix.de, rostedt@goodmis.org, joel@joelfernandes.org, sashal@kernel.org, daniel.vetter@ffwll.ch, duyuyang@gmail.com, johannes.berg@intel.com, tj@kernel.org, tytso@mit.edu, willy@infradead.org, david@fromorbit.com, amir73il@gmail.com, gregkh@linuxfoundation.org, kernel-team@lge.com, linux-mm@kvack.org, akpm@linux-foundation.org, mhocko@kernel.org, minchan@kernel.org, hannes@cmpxchg.org, vdavydov.dev@gmail.com, sj@kernel.org, jglisse@redhat.com, dennis@kernel.org, cl@linux.com, penberg@kernel.org, rientjes@google.com, vbabka@suse.cz, ngupta@vflare.org, linux-block@vger.kernel.org, josef@toxicpanda.com, linux-fsdevel@vger.kernel.org, jack@suse.cz, jlayton@kernel.org, dan.j.williams@intel.com, hch@infradead.org, djwong@kernel.org, dri-devel@lists.freedesktop.org, rodrigosiqueiramelo@gmail.com, melissa.srw@gmail.com, hamohammed.sa@gmail.com, harry.yoo@oracle.com, chris.p.wilson@intel.com, gwan-gyeong.mun@intel.com, max.byungchul.park@gmail.com, boqun.feng@gmail.com, longman@redhat.com, yskelg@gmail.com, yunseong.kim@ericsson.com, yeoreum.yun@arm.com, netdev@vger.kernel.org, matthew.brost@intel.com, her0gyugyu@gmail.com Subject: [PATCH v16 13/42] dept: apply sdt_might_sleep_{start,end}() to wait_for_completion()/complete() Date: Mon, 19 May 2025 18:17:57 +0900 Message-Id: <20250519091826.19752-14-byungchul@sk.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20250519091826.19752-1-byungchul@sk.com> References: <20250519091826.19752-1-byungchul@sk.com> X-Brightmail-Tracker: H4sIAAAAAAAAAzWSbUxTZxTH9zz39t5LZ/Xaod6BZqzxFSOKojnZpjHGZY8mbMaRJc4P8zpu druValpBWLKNjkK0CPGtoKKsFlOatghrTWRqCYOAMAPr8IrQIFM2iUSQCLQMYW4F45eTX845 /9/5cjhKe1uVwOmNRySTUTToGDWtHpnnXGecTJY39JQuh8jEMRou1voYCF31IvBds2AYavkI 7keHEUx3/E5BuT2E4PKjBxRca+1HEHT/yMDdv+eDEhlloN1ezEBBVS0DfzydwdBXdhqD158O f7oGabhz0omhfIiBivICHCtPMEy5PCy48lfAgPsCCzOPUqG9v1sFwfBaOF/Zx8CtYDsNrfUD GO7euMhAv+8/FdxpbaMhWpoIoVMlKqh55mTgadRFgSsyykJXowNDq2Mx1FljwqLxlyq4XdKI oejKzxiU3psIGo49xOD3dTPQHBnGEPDbKXhR3YJgoHSEhcITUyxUWEoRFBeW0WDt2wzT/8Qu X5pIBctPdTTU/NuNtm8lvkofIs3DoxSxBo6SF5F7DAlGHTT5zSmQXy48YIm1IcwShz+bBNzJ pOrWECaXxyIq4vccZ4h/7DRLbCMKJs86O9k9Sz9Xf5ApGfQ5kmn9tgNqebrJhg8HF+Y+uRdC +Sgw34biOIFPE5wtN6nXXHBiEM0yw68Senqm5vrxfJIQKBlU2ZCao/juN4X7l3rnlt7iZeGU vYqdZZpfITy+0hZjjtPwW4Si6ytfOd8RvHWNc564WDtc3DwX1fKbBcVbSc86Bb4iThhVXrKv Am8Lv7p76JNI40BveJBWb8zJEvWGtBQ5z6jPTfnyUJYfxd7L9d3M/no0Fvq0CfEc0s3T1AXX yFqVmGPOy2pCAkfp4jWewGpZq8kU876VTIe+MGUbJHMTSuRo3RLNxujRTC3/lXhE+kaSDkum 11PMxSXko2VtwarkFMb6mKOaTQcXTe7tqH6eJH8ibhW2LFqdF1bSf9jpsHTYupSMsc9qPuxa suPc2gXbJhzjSbkG847wROL6so74dHd/b+/XmQ2WNFKbsvv7eGVPtlK9a+b9xfJf19/L+HgS nx2ub9ugT9jnWXfGC+PLVoo1V1sKM97dtMDeqaPNspiaTJnM4v/IgWivWgMAAA== X-Brightmail-Tracker: H4sIAAAAAAAAAzWSa0yTZxTH9zzvlWrJa0V9I3FbOtkcDFaimBM1xLkYnyxx2TcTv8wqb2xj KaRVpCYq2EKwCAGTFpXLCjSVQbmsJRleylgZ1c6JXCooQxSCBCJKghRF8FIwfjn55fzP+Z0v h6cUvzMbea3+uGTQq3VKVkbLft5pTtS/iteobMUyCM8V0FDR7Gahp6kBgbs1F8NU1z4YnJ9G sHj3HgVlth4E1aOPKGgNjCDw1Z1joX88GkLhGRaCtkIWzLXNLPQ+W8IwbL+IocGzHx67Jmi4 U1KDoWyKhfIyM46USQwLrnoOXDlxMFZ3hYOl0WQIjgww0FkZZMA3lACXq4ZZuOkL0hBoG8PQ f72ChRH3ewbuBG7TMF8cCz2lRQw0vqhh4dm8iwJXeIaDvg4HhoBjPbRYItb8l+8YuFXUgSHf +QeG0MMbCNoLnmDwuAdY6AxPY/B6bBS8udqFYKz4OQd5FxY4KM8tRlCYZ6fBMpwCi68jlyvn kiH3txYaGt8OoN2pxF3lRqRzeoYiFu9J8iZ8nyW+eQdN/q0RybUrjzhiaR/iiMNzgnjr4knt zSlMqmfDDPHUn2eJZ/YiR6zPQ5i86O7mftl0ULYrTdJpsyTD96mHZJpFvxVn+tZkT97vQTnI G21FUbwobBPNFybQMrPCN+KDBwvUMscIX4reognGimQ8JQysEgcrH64MrRU0YqmtlltmWogT nzpvR5jn5cJ2Mf/Prz86vxAbWjpWPFGR9lBh58qqQkgRQw1VdAmSOdBn9ShGq89KV2t1KUnG YxqTXpuddCQj3YMiD+Q6vVTahub69/mRwCPlanmL71uNglFnGU3pfiTylDJGXu/dolHI09Sm U5Ih41fDCZ1k9KNYnlZukP90QDqkEI6qj0vHJClTMnxKMR+1MQflv8pzdv/N6kx7MlWXNs3G SRomsfXHrvH/3QmmJ18xaUOLTZ7RaWe7dLZ1whm95od1ZpWpuuRdwa5AwKo6nNWU2rg/ru1w KDaI24823zqTtGXr5b0laxNtvXfRjVVocLxg82QGcR+4/p89z96XvQNZ/+m2+PZGv/Zf/fw7 e4LqLyVt1KiT4ymDUf0B+5bOWzwDAAA= X-CFilter-Loop: Reflected Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Make dept able to track dependencies by wait_for_completion()/complete(). Signed-off-by: Byungchul Park --- include/linux/completion.h | 30 +++++++++++++++++++++++++----- 1 file changed, 25 insertions(+), 5 deletions(-) diff --git a/include/linux/completion.h b/include/linux/completion.h index fb2915676574..bd2c207481d6 100644 --- a/include/linux/completion.h +++ b/include/linux/completion.h @@ -10,6 +10,7 @@ */ =20 #include +#include =20 /* * struct completion - structure used to maintain state for a "completion" @@ -26,14 +27,33 @@ struct completion { unsigned int done; struct swait_queue_head wait; + struct dept_map dmap; }; =20 +#define init_completion(x) \ +do { \ + sdt_map_init(&(x)->dmap); \ + __init_completion(x); \ +} while (0) + +/* + * XXX: No use cases for now. Fill the body when needed. + */ #define init_completion_map(x, m) init_completion(x) -static inline void complete_acquire(struct completion *x) {} -static inline void complete_release(struct completion *x) {} + +static inline void complete_acquire(struct completion *x) +{ + sdt_might_sleep_start(&x->dmap); +} + +static inline void complete_release(struct completion *x) +{ + sdt_might_sleep_end(); +} =20 #define COMPLETION_INITIALIZER(work) \ - { 0, __SWAIT_QUEUE_HEAD_INITIALIZER((work).wait) } + { 0, __SWAIT_QUEUE_HEAD_INITIALIZER((work).wait), \ + .dmap =3D DEPT_MAP_INITIALIZER(work, NULL), } =20 #define COMPLETION_INITIALIZER_ONSTACK_MAP(work, map) \ (*({ init_completion_map(&(work), &(map)); &(work); })) @@ -75,13 +95,13 @@ static inline void complete_release(struct completion *= x) {} #endif =20 /** - * init_completion - Initialize a dynamically allocated completion + * __init_completion - Initialize a dynamically allocated completion * @x: pointer to completion structure that is to be initialized * * This inline function will initialize a dynamically created completion * structure. */ -static inline void init_completion(struct completion *x) +static inline void __init_completion(struct completion *x) { x->done =3D 0; init_swait_queue_head(&x->wait); --=20 2.17.1 From nobody Fri Dec 19 15:51:27 2025 Received: from invmail4.hynix.com (exvmail4.hynix.com [166.125.252.92]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 2A59626FD81; Mon, 19 May 2025 09:18:51 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=166.125.252.92 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747646335; cv=none; b=BiUFp9IzWNiw4rwwc9cQV30lHpHUuSvcHYvLxyqQJA8fO3BEp/62eyNp6S1mBS97aXfCOTw8wDGSMWtvgtInNnqNhQLL3aHozDivOwQvY/5Pi9pfPeqFNNmQV/fcgoalx52ulx5O3fbHzHEtI5X68GrQYKQRN6gMZ8IbKQsHD1w= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747646335; c=relaxed/simple; bh=pLUSXjgVOyB6ewzDMD2B/aTj2n2b3ii/EXLHyurcTzs=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References; b=BbQOt1wGFpa3aw7J9hcM9aw9C77CRbFe33i87S3DSNNlDyK0MpujLNDO+cTkQ6lt708FlMhs7D35aVvmCRu53edrkuU1eisVyRoBvPYRsu4aWTT5jRLnnQKXNmNnoE+zrdySflJ9CjhkWg5MmmntiLXsXO5LbRpHxDYfLXduCwM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com; spf=pass smtp.mailfrom=sk.com; arc=none smtp.client-ip=166.125.252.92 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=sk.com X-AuditID: a67dfc5b-681ff7000002311f-b5-682af76ebd89 From: Byungchul Park To: linux-kernel@vger.kernel.org Cc: kernel_team@skhynix.com, torvalds@linux-foundation.org, damien.lemoal@opensource.wdc.com, linux-ide@vger.kernel.org, adilger.kernel@dilger.ca, linux-ext4@vger.kernel.org, mingo@redhat.com, peterz@infradead.org, will@kernel.org, tglx@linutronix.de, rostedt@goodmis.org, joel@joelfernandes.org, sashal@kernel.org, daniel.vetter@ffwll.ch, duyuyang@gmail.com, johannes.berg@intel.com, tj@kernel.org, tytso@mit.edu, willy@infradead.org, david@fromorbit.com, amir73il@gmail.com, gregkh@linuxfoundation.org, kernel-team@lge.com, linux-mm@kvack.org, akpm@linux-foundation.org, mhocko@kernel.org, minchan@kernel.org, hannes@cmpxchg.org, vdavydov.dev@gmail.com, sj@kernel.org, jglisse@redhat.com, dennis@kernel.org, cl@linux.com, penberg@kernel.org, rientjes@google.com, vbabka@suse.cz, ngupta@vflare.org, linux-block@vger.kernel.org, josef@toxicpanda.com, linux-fsdevel@vger.kernel.org, jack@suse.cz, jlayton@kernel.org, dan.j.williams@intel.com, hch@infradead.org, djwong@kernel.org, dri-devel@lists.freedesktop.org, rodrigosiqueiramelo@gmail.com, melissa.srw@gmail.com, hamohammed.sa@gmail.com, harry.yoo@oracle.com, chris.p.wilson@intel.com, gwan-gyeong.mun@intel.com, max.byungchul.park@gmail.com, boqun.feng@gmail.com, longman@redhat.com, yskelg@gmail.com, yunseong.kim@ericsson.com, yeoreum.yun@arm.com, netdev@vger.kernel.org, matthew.brost@intel.com, her0gyugyu@gmail.com Subject: [PATCH v16 14/42] dept: apply sdt_might_sleep_{start,end}() to swait Date: Mon, 19 May 2025 18:17:58 +0900 Message-Id: <20250519091826.19752-15-byungchul@sk.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20250519091826.19752-1-byungchul@sk.com> References: <20250519091826.19752-1-byungchul@sk.com> X-Brightmail-Tracker: H4sIAAAAAAAAAz2Sa0yTdxTG/b/3VuveVDfeqVHSRLdhvNTocj6oMRHjP0aMcV+8zLgGXm0D FNMqt8QIE4yCBcQgXhBr3SrSSrF1mTqLWKSIBNZJKYVxGTjJiCDatcUiXgpRv5w8eZ7z/M6X w5HyJnoep9EeEnVaVYqCkVLS0VmmZdrxOPVK0/UlEAqeoKDCZmXAU2NBYL2VS8Bw42boDI8g eNP6JwnlZR4EVwZ6Sbjl7kPgrPqZgfZ/Z4M3NMZAc1khA8eu2hj46/kkAT1nSwmw2BOg3zxE QUuJiYDyYQYulh8jouM/AiLmahbMOYthsOoCC5MDSmju89Hg7F4K5yt7GLjnbKbAfXuQgPa7 FQz0Wd/T0OJ+REG4aD54ThtouPHCxMDzsJkEc2iMhSf1RgLcxq+gNi8KPP7/OxqaDPUEHP/l JgHerj8Q1J34hwC71cdAQ2iEAIe9jISJa40IBotGWcg/FWHhYm4RgsL8sxTk9ayBN6+jly8F lZB7uZaCG299aMM6bK20ItwwMkbiPEcGngh1MNgZNlL4sUnAdy70sjivrpvFRvth7KiKw1fv DRP4SiBEY3v1SQbbA6UsLhj1EvhFWxu7fcFu6dokMUWTLupWrP9Jqg4MdNAHG7lM/6932RzU xhQgCSfwq4X7+YbPusPmmtYM/43g90fIKT2XjxUchiG6AEk5kvfNFDovdaGpYA6fILzq7You cRzFLxbcNfFTtoz/XjhpaP7IXCRYauunOZKo313YMF2V82sEr6WSmmIKfLFEaDln/lj4WnhQ 5adKkMyIZlQjuUabnqrSpKxers7SajKXJ6al2lH0vcxHJvfcRgHPDy7Ec0gxS1br/E4tp1Xp +qxUFxI4UjFXVu34Vi2XJamyskVd2j7d4RRR70LzOUoRI1sVzkiS8wdUh8RkUTwo6j6lBCeZ l4Nqftv2haPTW5dY8Y7y3/n9qa9EwySnje/KcArKOkdChO/fatne3tW0Mf7hmdTsjSv3jj+V lB59SS4tDh7J8Py9Ux/cmxgbk325dc6iZ1++XFEQvwqfXlAcE7HRjf2B/TPX7ZwY6vjxcVEr ufAAH7tj0xYHr8u8pnMlB21o/fsnyl0KSq9WKeNInV71AcIr1Q9aAwAA X-Brightmail-Tracker: H4sIAAAAAAAAAzWSa0iTcRTG+793V6uXZfWiRDGKysg0Ug4ZFRX0FhRCRNGXGvrS5mXKpiuj QHNJzWYXUMvLWppr6cy5+aHbymbOVmim03SppVQkmpK5ee0ypb4cHp7nnN/z5TC45D4ZwiiU aYJKKUuSUiJCdCgme5NyIkweMTuzBHzjlwgoqbFQ0PqgCoGlLguDwcZ98N4/jGCm+S0Ohfmt CO709+JQ5+pD4DBfoKD982Lw+EYpcOfnUpBdXkPBu6FZDHoKbmBQZTsIH01fCXhzrQyDwkEK iguzscD4hsGUqZIGU+ZaGDAX0TDbHwnuvk4SGkrdJDi8G+GWoYeCpw43Aa6HAxi0Py6hoM/y h4Q3rlcE+PNCofW6noTqkTIKhvwmHEy+URra6o0YuIzLwaoNUHN+/iahSV+PQc7dWgw83U8Q PLv0CQObpZOCBt8wBnZbPg7T9xoRDOR9p+HilSkairPyEOReLCBA2xMFM5OB5tLxSMi6bSWg +lcn2rWDtxgsiG8YHsV5rf00P+3roHiH30jwr8s4/lFRL81rn3lp3mhL5+3mML786SDG3xnz kbyt8jLF28Zu0LzuuwfjR1pa6NiVx0Xb44UkhUZQbd5xUiQf6+8gUxuZM10Vj+lM1ELpUBDD sVu5jhrnvKbYdVxX1xQ+p4PZ1Zxd/5XUIRGDs50Lufel3WguWMoe5H70dgeWGIZg13KuB3vn bDEbzV3Wu/8xV3FV1vp5TlDA9+Y2zJ9K2CjOU2UgriGRES2oRMEKpSZZpkiKClcnyjOUijPh cSnJNhR4INP52esP0Xj7PidiGSRdJLY6NsglpEyjzkh2Io7BpcHiSvt6uUQcL8s4K6hSTqjS kwS1E4UyhHSF+MBR4aSEPSVLExIFIVVQ/U8xJigkE7VV+KuVwUXHorfcrI44SsQy21R1Vu2e iOnUwyPppQbPVU1MfIn5CT15vplsMhfIvet0z7XOn4zRszI39uXgtyhdYzmneZG2zDvUtCBh Z21PxqKcsf1m5Zf77KqJ6IrCc3kJzZ82HQnd45/sWKO/Wlyz1CKNiduNPmwJSZwwDPdKCbVc FhmGq9Syvzw9T588AwAA X-CFilter-Loop: Reflected Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Make dept able to track dependencies by swaits. Signed-off-by: Byungchul Park --- include/linux/swait.h | 3 +++ 1 file changed, 3 insertions(+) diff --git a/include/linux/swait.h b/include/linux/swait.h index d324419482a0..277ac74f61c3 100644 --- a/include/linux/swait.h +++ b/include/linux/swait.h @@ -6,6 +6,7 @@ #include #include #include +#include #include =20 /* @@ -161,6 +162,7 @@ extern void finish_swait(struct swait_queue_head *q, st= ruct swait_queue *wait); struct swait_queue __wait; \ long __ret =3D ret; \ \ + sdt_might_sleep_start(NULL); \ INIT_LIST_HEAD(&__wait.task_list); \ for (;;) { \ long __int =3D prepare_to_swait_event(&wq, &__wait, state);\ @@ -176,6 +178,7 @@ extern void finish_swait(struct swait_queue_head *q, st= ruct swait_queue *wait); cmd; \ } \ finish_swait(&wq, &__wait); \ + sdt_might_sleep_end(); \ __out: __ret; \ }) =20 --=20 2.17.1 From nobody Fri Dec 19 15:51:27 2025 Received: from invmail4.hynix.com (exvmail4.hynix.com [166.125.252.92]) by smtp.subspace.kernel.org (Postfix) with ESMTP id A76152701C6; Mon, 19 May 2025 09:18:51 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=166.125.252.92 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747646335; cv=none; b=Q3HLxXqMkI3FmHFkpu3rmJG/0hpfTpPRPotu/SAtDuIXT+606dz+Yh4nP5/V5Sp8yxfeQEn+42t8AdOhnlIYfnwJIXLNRfaDDoxeAyY6X5dyTTwz5ae3FTcnIPabvPDD7ccFqdRmkr+cjSeRieO/x0Ika+lyTv6uepLbgSQ6Saw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747646335; c=relaxed/simple; bh=l20SQrf5lRl3+sKTvdmaOt5bA5DWEDLDzGURyc0sECY=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References; b=aljxxVpMCt1l+Gxdsdm32yxFdU49ukXcNAeMXJOHSimSM7X4UtOWpS47O++7usbl62WWC5+ADT2F4o7B38NY7F9etcvgxo97G1+p4NohYfeHq48QzuUjV/zw9Qa8slH4ha25YkEpGfLVye+Ww5aNwPN0Qg1B1ICZFOHxcopkNI0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com; spf=pass smtp.mailfrom=sk.com; arc=none smtp.client-ip=166.125.252.92 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=sk.com X-AuditID: a67dfc5b-681ff7000002311f-c5-682af76e4861 From: Byungchul Park To: linux-kernel@vger.kernel.org Cc: kernel_team@skhynix.com, torvalds@linux-foundation.org, damien.lemoal@opensource.wdc.com, linux-ide@vger.kernel.org, adilger.kernel@dilger.ca, linux-ext4@vger.kernel.org, mingo@redhat.com, peterz@infradead.org, will@kernel.org, tglx@linutronix.de, rostedt@goodmis.org, joel@joelfernandes.org, sashal@kernel.org, daniel.vetter@ffwll.ch, duyuyang@gmail.com, johannes.berg@intel.com, tj@kernel.org, tytso@mit.edu, willy@infradead.org, david@fromorbit.com, amir73il@gmail.com, gregkh@linuxfoundation.org, kernel-team@lge.com, linux-mm@kvack.org, akpm@linux-foundation.org, mhocko@kernel.org, minchan@kernel.org, hannes@cmpxchg.org, vdavydov.dev@gmail.com, sj@kernel.org, jglisse@redhat.com, dennis@kernel.org, cl@linux.com, penberg@kernel.org, rientjes@google.com, vbabka@suse.cz, ngupta@vflare.org, linux-block@vger.kernel.org, josef@toxicpanda.com, linux-fsdevel@vger.kernel.org, jack@suse.cz, jlayton@kernel.org, dan.j.williams@intel.com, hch@infradead.org, djwong@kernel.org, dri-devel@lists.freedesktop.org, rodrigosiqueiramelo@gmail.com, melissa.srw@gmail.com, hamohammed.sa@gmail.com, harry.yoo@oracle.com, chris.p.wilson@intel.com, gwan-gyeong.mun@intel.com, max.byungchul.park@gmail.com, boqun.feng@gmail.com, longman@redhat.com, yskelg@gmail.com, yunseong.kim@ericsson.com, yeoreum.yun@arm.com, netdev@vger.kernel.org, matthew.brost@intel.com, her0gyugyu@gmail.com Subject: [PATCH v16 15/42] dept: apply sdt_might_sleep_{start,end}() to waitqueue wait Date: Mon, 19 May 2025 18:17:59 +0900 Message-Id: <20250519091826.19752-16-byungchul@sk.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20250519091826.19752-1-byungchul@sk.com> References: <20250519091826.19752-1-byungchul@sk.com> X-Brightmail-Tracker: H4sIAAAAAAAAAzWSa0yTZxSAfd/v2mK3L1XnNyXRdBoVFcWgO1nQkZlsry5OkUQT/aGNfLGN pbJWkc6QgAJxMJjTAFEutrDV2latLYl4KakQOxhRsdSChIsSR2RycYxWUbwUnH9OnpzLc86P w1NKPzOP1+oPSQa9Wqdi5bR8eKZlpf5FnGZ1fv0KCI+foKHyspOFtksOBM66XAyDt7+DjsgQ gtd37lFQXtqGwPK4h4I6fy8Cr+0YC+1PPoFgeJSFltIiFo7XXmbh/rNJDN1lpzA43FugzzpA Q+vJGgzlgyxUlB/H0fAUw4TVzoE1ZzH0285yMPk4AVp6Qwx4u5bDmepuFm56W2jw1/djaL9e yUKv8x0Drf5mGiIl86Htt2IGLo7UsPAsYqXAGh7lIOAzY/CbPwNXXlRY8N9bBv4s9mEo+P0K huDDGwgaTjzC4HaGWGgKD2HwuEspeHX+NoL+kmEO8n+Z4KAitwRBUX4ZDXnda+H1y+jmqvEE yD3nouHimxBKXk+c1U5EmoZGKZLnOUJehR+wxBsx0+SvGpFcO9vDkbyGLo6Y3YeJxxZHam8O YmIZCzPEbf+ZJe6xUxwpHA5iMnL3Lrctdpc8KU3SaTMlw6oNe+Wahp60jBJZ1pA/QOWgYq4Q yXhRSBSDj66ij+w68wc1xaywROzsnJjm2cJC0VM8wBQiOU8JoRixo+rh9MAsYYfY/tRHTzEt LBY7Ry9Em3heIawT6+zxH5wLRIfLN+2RRdNdRU3To0phrRh0VNNTTlE4LRMDvtD/R3wu3rJ1 0ieRwoxm2JFSq89MV2t1ifEak16bFb/vYLobRb/Lmj25ux6NtaU2IoFHqpkKl3eZRsmoM42m 9EYk8pRqtsLuWapRKtLUpp8kw8E9hsM6ydiI5vO0aq5iTeRImlLYrz4kHZCkDMnwsYp52bwc tEl39asf+57LkjvCX8dyiwJZ+dqCNaWLyLckJsb198I5hSnoyaftqdjYzeAvfsjeak7ZzHr7 Ng0kJrc6mn1Jq9bNmPWvL2lvmWXjS9XcPQf+MSkzuPGt/bHfbOxaZuVqX6QsaM5ekYoSKmyn A3KsybpiqTSNfPm9rGH7Pubozl+rVLRRo06IowxG9XttCfiSWQMAAA== X-Brightmail-Tracker: H4sIAAAAAAAAAzWSe0hTcRTH+923q8VtWl2syAb2sPJRGieKnkS3QosIhUhq5aWN5orNLINA 08TUSQlz9NDma825am3SyybD4cwss7ZMxVlJRJIl2Lay7DGN/jl8ON9zPuefw+ASExnJKFRZ glolU0opESFKWZe/UvUtRh7/pnM2BPxFBFy7baGg+1YjAktTHgbDbdvhdXAEwc9nz3HQ67oR VL/z4dDkHkTgMJ2jwPN+JngDoxR06EooyK+9TcGLTxMYDFSUY9BoS4Y3xg8EdF6swUA/TMFV fT4WKh8xGDeaaTDmRsOQ6QoNE+8SoGOwhwRXZQcJjv7lcLlqgIJHjg4C3PeHMPA8vEbBoOUP CZ3uxwQEy+ZB9yUtCTe/1FDwKWjEwRgYpeGl04CB2zAHrAUha+HX3yS0a50YFNbdwcDb14yg pegtBjZLDwWuwAgGdpsOhx832hAMlX2m4XzpOA1X88oQlJyvIKBgIAl+fg9drvQnQN51KwE3 f/WgTRt4S5UF8a6RUZwvsJ/ifwReUbwjaCD4JzUc/+CKj+YLWvpp3mA7ydtNMXzto2GMrx4L kLzNfIHibWPlNF/82YvxX7q66D0L9ovWZwhKRbagjttwSCRv8WWcKAs7PeJ+ieciLV2MwhiO TeSsl+vxSabYJVxv7/gUR7BRnF37gSxGIgZne6Zzryv70GQQzqZyno9OYpIJNprrHW0IDTGM mF3DNZlj/zkXco1W55QnLNTuL3FNrUrYJM7bWEVcRCIDmmZGEQpVdqZMoUyK1RyT56gUp2OP HM+0odD/GM9OXLqP/J7trYhlkHSG2OpYJpeQsmxNTmYr4hhcGiE225fKJeIMWc4ZQX38oPqk UtC0onkMIZ0r3pkmHJKwR2VZwjFBOCGo/6cYExaZi6Lewi6dUpteFNcu949nn4t3Dexe/j3N qVAcjW+O3n24q35Vije/QhVF1a7cWxiZfsArvZuFGpJ18zcvS6tPTr3nW/urdPrdxYGHqat9 iTn7oG5HW+rG8qyipx69xmx6sQ3fsmNW/1Z9ZPOimoanEkliur96rO966au4GcHwTSvipIRG LkuIwdUa2V860ChtOwMAAA== X-CFilter-Loop: Reflected Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Make dept able to track dependencies by waitqueue waits. Signed-off-by: Byungchul Park --- include/linux/wait.h | 3 +++ 1 file changed, 3 insertions(+) diff --git a/include/linux/wait.h b/include/linux/wait.h index 965a19809c7e..aae161e500dc 100644 --- a/include/linux/wait.h +++ b/include/linux/wait.h @@ -7,6 +7,7 @@ #include #include #include +#include =20 #include =20 @@ -303,6 +304,7 @@ extern void init_wait_entry(struct wait_queue_entry *wq= _entry, int flags); struct wait_queue_entry __wq_entry; \ long __ret =3D ret; /* explicit shadow */ \ \ + sdt_might_sleep_start(NULL); \ init_wait_entry(&__wq_entry, exclusive ? WQ_FLAG_EXCLUSIVE : 0); \ for (;;) { \ long __int =3D prepare_to_wait_event(&wq_head, &__wq_entry, state);\ @@ -321,6 +323,7 @@ extern void init_wait_entry(struct wait_queue_entry *wq= _entry, int flags); break; \ } \ finish_wait(&wq_head, &__wq_entry); \ + sdt_might_sleep_end(); \ __out: __ret; \ }) =20 --=20 2.17.1 From nobody Fri Dec 19 15:51:27 2025 Received: from invmail4.hynix.com (exvmail4.hynix.com [166.125.252.92]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 8EDB52701C4; Mon, 19 May 2025 09:18:51 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=166.125.252.92 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747646335; cv=none; b=MLgJ4lumi8CBOGeAa553swn0ZNmZE7kIwAvhLoHZoSeUQYnxQM5lDfwSm4tykOltGDEuNHrURFqM1vlvNfY/18iYLZjlkVyO2HbB/4wYd9CjMQxsXs+T2ZgobE2FiEqXPbEUzjdJAFtbx+RFb2kKpDOSs1NgYZl0ptvC7pWx9yA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747646335; c=relaxed/simple; bh=+bd9WiorYKiALUTe/ZE9Twmt5uX3p97amk10CfbvR3w=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References; b=DSHG+3jzSq5mxotQcYjT2obxH5frrQ7GNMrZDCPb0jxJdZqpWinzmXCYMzEL1d2sJMjA/AhB+rHQZZ8slghV/EPukDexyWiEO1aMx2w7IjsTJu4hzDgDHdzteNa/V7dj4dRnMoWKq/SFN1cjwHhLpf5Q1mZWqYXXIxsecSamJiI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com; spf=pass smtp.mailfrom=sk.com; arc=none smtp.client-ip=166.125.252.92 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=sk.com X-AuditID: a67dfc5b-681ff7000002311f-d5-682af76ec40c From: Byungchul Park To: linux-kernel@vger.kernel.org Cc: kernel_team@skhynix.com, torvalds@linux-foundation.org, damien.lemoal@opensource.wdc.com, linux-ide@vger.kernel.org, adilger.kernel@dilger.ca, linux-ext4@vger.kernel.org, mingo@redhat.com, peterz@infradead.org, will@kernel.org, tglx@linutronix.de, rostedt@goodmis.org, joel@joelfernandes.org, sashal@kernel.org, daniel.vetter@ffwll.ch, duyuyang@gmail.com, johannes.berg@intel.com, tj@kernel.org, tytso@mit.edu, willy@infradead.org, david@fromorbit.com, amir73il@gmail.com, gregkh@linuxfoundation.org, kernel-team@lge.com, linux-mm@kvack.org, akpm@linux-foundation.org, mhocko@kernel.org, minchan@kernel.org, hannes@cmpxchg.org, vdavydov.dev@gmail.com, sj@kernel.org, jglisse@redhat.com, dennis@kernel.org, cl@linux.com, penberg@kernel.org, rientjes@google.com, vbabka@suse.cz, ngupta@vflare.org, linux-block@vger.kernel.org, josef@toxicpanda.com, linux-fsdevel@vger.kernel.org, jack@suse.cz, jlayton@kernel.org, dan.j.williams@intel.com, hch@infradead.org, djwong@kernel.org, dri-devel@lists.freedesktop.org, rodrigosiqueiramelo@gmail.com, melissa.srw@gmail.com, hamohammed.sa@gmail.com, harry.yoo@oracle.com, chris.p.wilson@intel.com, gwan-gyeong.mun@intel.com, max.byungchul.park@gmail.com, boqun.feng@gmail.com, longman@redhat.com, yskelg@gmail.com, yunseong.kim@ericsson.com, yeoreum.yun@arm.com, netdev@vger.kernel.org, matthew.brost@intel.com, her0gyugyu@gmail.com Subject: [PATCH v16 16/42] dept: apply sdt_might_sleep_{start,end}() to hashed-waitqueue wait Date: Mon, 19 May 2025 18:18:00 +0900 Message-Id: <20250519091826.19752-17-byungchul@sk.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20250519091826.19752-1-byungchul@sk.com> References: <20250519091826.19752-1-byungchul@sk.com> X-Brightmail-Tracker: H4sIAAAAAAAAAzWSa0hTcRjG+5/7VqvTsjxdoBhYaXRZN1/CIvpQByK6fcrosvLUVrpkNi+B oKW1vKWGSqU2NZZtK/XMIKvZcmXZxVaaqehME0nUJGsr0y5z1ZeXh+d539/z5WVw+RNyDqPR nhR0WlWkgpIS0qEpJUu130LUK9xiKHi+GggorLBS4LplQWCtTsag//EWeOcdRDD28hUOBXku BCXdnThU17sR2MtPU9DUOxWaPcMUNOSlU3CmrIKC1wPjGHTk52JgEbdBl6mPgOfZpRgU9FNw peAM5hsfMRg1mWkwJQVBT/llGsa7ldDgbiHB3r4ELhV3UHDf3kBA/Z0eDJruFlLgtv4m4Xn9 UwK8WXPBlZNJws1PpRQMeE04mDzDNLxxGDGoN86CyhQf8OyXXyQ8yXRgcPZaFQbNbfcQ1Bre YyBaWyhwegYxsIl5OPy4/hhBT9YQDakZozRcSc5CkJ6aT0BKxxoY++5rLvqqhOSrlQTc/NmC Nq7nrcVWxDsHh3E+xRbH//C8pXi710jwz0o5vuZyJ82n1LbTvFHU87byEL7sfj/Gl4x4SF40 n6d4cSSX5tOGmjH+U2MjvWNeuDQsQojUxAq65RsOStXVL86j6C4mvsxVgSehLioNSRiOXc0l v3EQaYjx63OvmQmbYhdxra2j+IQOYBdwtsw+Mg1JGZxtmcy9K2pDE8EMdh93u6/QzyHYIO7h C4vfl7FruTb3BeIvfz5nqXT4QRKf357u9O/I2TVcs6WYmIBy7EUJl2Fu/3cwm3tY3kpkI5kR TTIjuUYbG6XSRK5epk7QauKXHT4RJSLfe5kSx/feQSOu3XWIZZBiiqzSHqyWk6rYmISoOsQx uCJAZrYtVstlEaqEU4LuxAGdPlKIqUNzGUIRKFvpjYuQs0dVJ4XjghAt6P6nGCOZk4QODb1f mdH0c5XJvbOA3KW/pe6eduzZWCAzFjTDpT0epHIEdm4K2xOdfyM80TnyISfUPPnisWBDaNvW gYCiR8rY2u3Zyld43PTNOevs1pqnm2f2pAqWKDGscfRQ35HEOLZq/8aFUw0RAZKi+JwmffSD z5se1fQazoX2tjov5C6QrdIriBi1ShmC62JUfwB3y8NxWgMAAA== X-Brightmail-Tracker: H4sIAAAAAAAAAzWSeUiTcRzG+723y9XbMntJshrZsegQMr7QQf/50kUIEQSVQ1/aaq7a8oxA Uzs0zQK1dK6lsmybaZuBHRNztDStZk5T8yamomaHM007ptA/Dx+e58vz/PNlcMlDcgWjVF8Q NGq5SkqJCNGhnamb1T9lim3P6oPAO3GNAF2FhQLXIzMCS1UKBsOvwuHj5CiCmbfvccjPdSG4 39+NQ5WzB4G97DIFLZ8Xgds7TkFDbiYFqSUVFDSPzGLQlXcbA7P1IPQaPQQ05hRjkD9MQWF+ KuaTIQymjSYajMkhMFBWQMNsfyg09LSR4ChqIMHeuQnu6rsoeGFvIMBZPYBByzMdBT2WvyQ0 OusJmMwOAtetLBLKvxRTMDJpxMHoHafhQ60BA6chECrTfK1Xfvwh4XVWLQZXSh9j4O54jqDm Wh8GVksbBQ7vKAY2ay4Ovx68QjCQPUZD+o1pGgpTshFkpucRkNYVBjNTvuWiiVBIuVdJQPnv NrR3D2/RWxDvGB3H+TRbPP/L20rx9kkDwb8p5vinBd00n1bTSfMGayxvK5PxJS+GMf7+dy/J W03XKd76/TbNZ4y5Mf7Lu3f04ZXHRLuiBZUyTtBs3RMpUlQ1XUfnepmEElcFnox6qQzEMBy7 nbvazGQgP4Zi13Pt7dP4HAewqzlblofMQCIGZ9sWch+LOtBcsJQ9zj3x6Kg5JtgQ7mWTed4X szu4jp6bxBxz7CrOXFk7X+Tn8zszHfM3EjaMc5v1RA4SGdACEwpQquNi5EpV2BbtGUWiWpmw JepsjBX5Hsh4afZWNZpoCa9DLIOk/uJK+0aFhJTHaRNj6hDH4NIAscm2QSERR8sTkwTN2ZOa WJWgrUNBDCFdLt53VIiUsKfkF4QzgnBO0PxPMcZvRTLaHfSJjL/U7zwhK22tGxpsX7tpcdN+ pi80OOe0o0q8P5kpPX/6W9JVYdVMju5AY+GJY10B8RPu2fB1g3f8GUOwalnekdTADVO7Wwvd hkbTyHJdRLNLfzE+SuEfQn5dsKagfHBJua6793e9Xlo2HrH4TkTN3jHlZVmSyzPtCda+TpcS WoU8VIZrtPJ/tuvmajwDAAA= X-CFilter-Loop: Reflected Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Make dept able to track dependencies by hashed-waitqueue waits. Signed-off-by: Byungchul Park --- include/linux/wait_bit.h | 3 +++ 1 file changed, 3 insertions(+) diff --git a/include/linux/wait_bit.h b/include/linux/wait_bit.h index 9e29d79fc790..179a616ad245 100644 --- a/include/linux/wait_bit.h +++ b/include/linux/wait_bit.h @@ -6,6 +6,7 @@ * Linux wait-bit related types and methods: */ #include +#include =20 struct wait_bit_key { unsigned long *flags; @@ -257,6 +258,7 @@ extern wait_queue_head_t *__var_waitqueue(void *p); struct wait_bit_queue_entry __wbq_entry; \ long __ret =3D ret; /* explicit shadow */ \ \ + sdt_might_sleep_start(NULL); \ init_wait_var_entry(&__wbq_entry, var, \ exclusive ? WQ_FLAG_EXCLUSIVE : 0); \ for (;;) { \ @@ -274,6 +276,7 @@ extern wait_queue_head_t *__var_waitqueue(void *p); cmd; \ } \ finish_wait(__wq_head, &__wbq_entry.wq_entry); \ + sdt_might_sleep_end(); \ __out: __ret; \ }) =20 --=20 2.17.1 From nobody Fri Dec 19 15:51:27 2025 Received: from invmail4.hynix.com (exvmail4.hynix.com [166.125.252.92]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 2E2522741D6; Mon, 19 May 2025 09:18:54 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=166.125.252.92 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747646337; cv=none; b=J6bTlB5lUe+xVK2IIdWmeaNNNKl4D80kECGxs1Q2lKaBv4N70jleTXlqtjNP4dtD0lKOhWqUE2Tuu0FvK/0CCvi+qkdQZHJ4p/tO1EsOKJk0vpm8FlDwVX3S4IFxZFXgj5IZzngXk1RF7LLQko6Ij2hzWJaXBFS7t7GNWLML7x8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747646337; c=relaxed/simple; bh=9Sbtv7YuVMjEizoot53ogfQz7NbvVKio90qc+0mHg80=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References; b=osAygqQY4awYv9YDRf4IMYh8WqtzwG68wLWCE2wbZqXwhk6nwtFlHp6yVcFIX6uvW/i8pyeWwyt/Lj4OLVh6wQr8YQytQWvpuQkhFYr5Brh8+9cuVm8svrJTmcBRNuNqa9nwfl1JVqSuzp1Rjn3uiqBzM2LdnUoyiX6/lNtHc+g= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com; spf=pass smtp.mailfrom=sk.com; arc=none smtp.client-ip=166.125.252.92 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=sk.com X-AuditID: a67dfc5b-681ff7000002311f-e4-682af76e80fc From: Byungchul Park To: linux-kernel@vger.kernel.org Cc: kernel_team@skhynix.com, torvalds@linux-foundation.org, damien.lemoal@opensource.wdc.com, linux-ide@vger.kernel.org, adilger.kernel@dilger.ca, linux-ext4@vger.kernel.org, mingo@redhat.com, peterz@infradead.org, will@kernel.org, tglx@linutronix.de, rostedt@goodmis.org, joel@joelfernandes.org, sashal@kernel.org, daniel.vetter@ffwll.ch, duyuyang@gmail.com, johannes.berg@intel.com, tj@kernel.org, tytso@mit.edu, willy@infradead.org, david@fromorbit.com, amir73il@gmail.com, gregkh@linuxfoundation.org, kernel-team@lge.com, linux-mm@kvack.org, akpm@linux-foundation.org, mhocko@kernel.org, minchan@kernel.org, hannes@cmpxchg.org, vdavydov.dev@gmail.com, sj@kernel.org, jglisse@redhat.com, dennis@kernel.org, cl@linux.com, penberg@kernel.org, rientjes@google.com, vbabka@suse.cz, ngupta@vflare.org, linux-block@vger.kernel.org, josef@toxicpanda.com, linux-fsdevel@vger.kernel.org, jack@suse.cz, jlayton@kernel.org, dan.j.williams@intel.com, hch@infradead.org, djwong@kernel.org, dri-devel@lists.freedesktop.org, rodrigosiqueiramelo@gmail.com, melissa.srw@gmail.com, hamohammed.sa@gmail.com, harry.yoo@oracle.com, chris.p.wilson@intel.com, gwan-gyeong.mun@intel.com, max.byungchul.park@gmail.com, boqun.feng@gmail.com, longman@redhat.com, yskelg@gmail.com, yunseong.kim@ericsson.com, yeoreum.yun@arm.com, netdev@vger.kernel.org, matthew.brost@intel.com, her0gyugyu@gmail.com Subject: [PATCH v16 17/42] dept: apply sdt_might_sleep_{start,end}() to dma fence Date: Mon, 19 May 2025 18:18:01 +0900 Message-Id: <20250519091826.19752-18-byungchul@sk.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20250519091826.19752-1-byungchul@sk.com> References: <20250519091826.19752-1-byungchul@sk.com> X-Brightmail-Tracker: H4sIAAAAAAAAAzWSbUxTZxTHfe69vfe2o3rtnLvTJWod6NiGYmA5BjX7Ynw+uGSLiYnug1a5 2ma0kPJeMwOzKiuCikF8oV0pWpu2AraNQaHASujsCK6bWKBDVGJUYmsTRutAfGln/HLyy/mf 8ztfDkvK/KJlrEpTImg1igI5LaEk0bSWrzQvMpXrY715EJ+poaC53UlDsM2BwOmpJmBqYBuM JCIIXg79SUJTYxBBy8N7JHj8Ewi8tp9puPNoIQzHYzQEGmtpONLaTsNfz+YJGD/bQIDD9S3c tz6mYPCUhYCmKRouNh0hkuUpAbNWOwPWqnSYtF1gYP5hNgQmQiLwhr+A86ZxGrq9AQr8nZME 3LnZTMOE840IBv23KEjUL4fg6ToRXH1uoeFZwkqCNR5j4O8+MwF+81Lo0CeFx/59LYLf6/oI OHbpGgHDY10IemoeEOByhmjoj0cIcLsaSZi7MoBgsj7KwNETswxcrK5HUHv0LAX68Vx4+V/y snEmG6p/7aDg6qsQ+mYzdpqcCPdHYiTWu8vxXPwujb0JM4X/sPD4xoV7DNb3hBlsdpVity0T t3ZPEbhlOi7CLvsvNHZNNzDYEB0m8PPbt5nvPt0t2ZQvFKjKBO26LXslyq7jl8miZmnFU89J pgrNSwxIzPJcDu8zGpn33Hs3TKWY5tbwo6OzZIqXcCt5d91jkQFJWJILfcCPGMeQAbHsh9z3 fPR4XgopLp0/N5CWGpdyX/PG1h7ynXIF7+jo+5/FyX64th+lWMbl8sMOE5VS8twZMW8ZHKHe LXzC/2YbpU4hqRktsCOZSlOmVqgKcrKUlRpVRdb+QrULJb/L+tP8D51oOrjDhzgWydOkHd7P lTKRoqy4Uu1DPEvKl0jt7rVKmTRfUakTtIV7tKUFQrEPLWcp+cfSDYnyfBl3UFEi/CgIRYL2 fUqw4mVVSL2x3LY6sPWkmIvq/dmmotgTz6t/8MqaQs1ncupLW++NXQ2bRvY5HHOd16srDqhN JQvCi7e3zugWh3RDteflQ+HC8KrEFnNO97pbhhr3oxUvDmXhNt2qvMj1gxken/RwYFfCaJHr cs8FQ1t3dnHjGZtDh+0ftY1lRPaoF5VuyJBTxUpFdiapLVa8BfVWACVZAwAA X-Brightmail-Tracker: H4sIAAAAAAAAAzXSf0zMcRgHcJ/v774c350b3/m9MyREyJ6pYTZ8/Q5/2Pgjt/rOna7L7ogY 6+ryo1x+bDlSuYpz6tK5axa5tHIpDVGKlrhmVvTD6OL6gTvmn2evPc+z9/PPw+BSKzmFUWkO iVqNQi2nWILdFpG6SPMjRLnE0DUbvANnCMgptVHQeKcYga1Mj0G3ewO0DvYgGH72AgdTViOC fM87HMpqOxC4rCkUNH0cD83efgrqszIoSC0speDllxEM2i9fwqDYsRXeWz4R0HChAANTNwXX TKmYv3Rh4LMU0WBJngOd1mwaRjxhUN/RQkJNbj0JrrYFcDWvnYKHrnoCass7MWh6kENBh+03 CQ21dQQMZk6FxotGEkr6Cij4MmjBweLtp+FVlRmDWvMksBv8qae+/yLhibEKg1M37mLQ/LYC QeWZDxg4bC0U1Hh7MHA6snAYuuVG0JnZS0PaOR8N1/SZCDLSLhNgaA+H4Z/+y7kDYaC/bieg ZLQFrVkl2PJsSKjp6ccFg/OIMOR9TQmuQTMhPC3ghfvZ72jBUNlGC2bHYcFpDREKH3ZjQv43 Lyk4is5SguPbJVpI723GhL7nz+mo6XvYyFhRrUoUtYtX7WOVFadv4gdzJEe7ys7TyWiETUdB DM8t5x+9biMCprh5/Js3PjxgGTeLdxo/kemIZXCuZSzfmvsWpSOGmcjt4HtPRwRIcHP4K+5x gXUJt4LPLazE/0XO5IvtVX8d5O+3ZdSggKVcON9cnEdcQKwZjSlCMpUmMV6hUoeH6uKUSRrV 0dCYhHgH8v+P5cTIxXI00LShGnEMko+T2F3zlVJSkahLiq9GPIPLZZIiZ7BSKolVJB0TtQnR 2sNqUVeNpjKEfLJk025xn5TbrzgkxoniQVH7f4oxQVOS0Zhd78uHNpfM6OVY92S90zTps7J8 7rIGo4KLMWxZmRARkpfhC4+MOe7rz/9R53GtDYXtE66cc52Nim5iZnl00tEE9XDf3nurHxdM G/VUVbjXd0YciHwgW3DyxcY0tiNlaUpclsmOvurGbm99ykXpox3Z684ncrd/b9spM6qtwQvl hE6pCAvBtTrFHwJcWHs7AwAA X-CFilter-Loop: Reflected Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Make dept able to track dependencies by dma fence waits and signals. Signed-off-by: Byungchul Park --- drivers/dma-buf/dma-fence.c | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/drivers/dma-buf/dma-fence.c b/drivers/dma-buf/dma-fence.c index f0cdd3e99d36..5d2fd2f6a46d 100644 --- a/drivers/dma-buf/dma-fence.c +++ b/drivers/dma-buf/dma-fence.c @@ -16,6 +16,7 @@ #include #include #include +#include =20 #define CREATE_TRACE_POINTS #include @@ -783,6 +784,7 @@ dma_fence_default_wait(struct dma_fence *fence, bool in= tr, signed long timeout) cb.task =3D current; list_add(&cb.base.node, &fence->cb_list); =20 + sdt_might_sleep_start(NULL); while (!test_bit(DMA_FENCE_FLAG_SIGNALED_BIT, &fence->flags) && ret > 0) { if (intr) __set_current_state(TASK_INTERRUPTIBLE); @@ -796,6 +798,7 @@ dma_fence_default_wait(struct dma_fence *fence, bool in= tr, signed long timeout) if (ret > 0 && intr && signal_pending(current)) ret =3D -ERESTARTSYS; } + sdt_might_sleep_end(); =20 if (!list_empty(&cb.base.node)) list_del(&cb.base.node); @@ -885,6 +888,7 @@ dma_fence_wait_any_timeout(struct dma_fence **fences, u= int32_t count, } } =20 + sdt_might_sleep_start(NULL); while (ret > 0) { if (intr) set_current_state(TASK_INTERRUPTIBLE); @@ -899,6 +903,7 @@ dma_fence_wait_any_timeout(struct dma_fence **fences, u= int32_t count, if (ret > 0 && intr && signal_pending(current)) ret =3D -ERESTARTSYS; } + sdt_might_sleep_end(); =20 __set_current_state(TASK_RUNNING); =20 --=20 2.17.1 From nobody Fri Dec 19 15:51:27 2025 Received: from invmail4.hynix.com (exvmail4.hynix.com [166.125.252.92]) by smtp.subspace.kernel.org (Postfix) with ESMTP id B69332749E8; Mon, 19 May 2025 09:18:54 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=166.125.252.92 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747646339; cv=none; b=U2/lW4yMeufnOpoPQSP3Ad6eWkSodidrS+Q/ncban9rVauITNxA7gh4cy1g2JTpGL06+aQrG2IwGM9yOQH5rxy1iowgkqLglNXSgfl1SP5u3bE/XG1R2u/xHCdd0s3pjlBnGMVDx7SrVLomagluVsdPDZUrPF01CrE2MVllkZ/E= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747646339; c=relaxed/simple; bh=q58Uga80B34rBaKlkqZqhMJuk4jxvsWlEkIvfJAN07I=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References; b=P1oxQRcXegIU4ZvMYe7b2NjX4jxCbOclztJThv69Qsb8Vv1d4zlwChgrhgVcyoS8gQ1dFHlJMmK/4reK6ZyrfgdBbA5vnLp2BmDtN0+wfsahb2VJ0xoyiIiQPcvalCbtqafh5bSQ7Z5hVKu1MtebfB+MHzxaeKp9zw3ozutB780= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com; spf=pass smtp.mailfrom=sk.com; arc=none smtp.client-ip=166.125.252.92 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=sk.com X-AuditID: a67dfc5b-681ff7000002311f-f5-682af76ed623 From: Byungchul Park To: linux-kernel@vger.kernel.org Cc: kernel_team@skhynix.com, torvalds@linux-foundation.org, damien.lemoal@opensource.wdc.com, linux-ide@vger.kernel.org, adilger.kernel@dilger.ca, linux-ext4@vger.kernel.org, mingo@redhat.com, peterz@infradead.org, will@kernel.org, tglx@linutronix.de, rostedt@goodmis.org, joel@joelfernandes.org, sashal@kernel.org, daniel.vetter@ffwll.ch, duyuyang@gmail.com, johannes.berg@intel.com, tj@kernel.org, tytso@mit.edu, willy@infradead.org, david@fromorbit.com, amir73il@gmail.com, gregkh@linuxfoundation.org, kernel-team@lge.com, linux-mm@kvack.org, akpm@linux-foundation.org, mhocko@kernel.org, minchan@kernel.org, hannes@cmpxchg.org, vdavydov.dev@gmail.com, sj@kernel.org, jglisse@redhat.com, dennis@kernel.org, cl@linux.com, penberg@kernel.org, rientjes@google.com, vbabka@suse.cz, ngupta@vflare.org, linux-block@vger.kernel.org, josef@toxicpanda.com, linux-fsdevel@vger.kernel.org, jack@suse.cz, jlayton@kernel.org, dan.j.williams@intel.com, hch@infradead.org, djwong@kernel.org, dri-devel@lists.freedesktop.org, rodrigosiqueiramelo@gmail.com, melissa.srw@gmail.com, hamohammed.sa@gmail.com, harry.yoo@oracle.com, chris.p.wilson@intel.com, gwan-gyeong.mun@intel.com, max.byungchul.park@gmail.com, boqun.feng@gmail.com, longman@redhat.com, yskelg@gmail.com, yunseong.kim@ericsson.com, yeoreum.yun@arm.com, netdev@vger.kernel.org, matthew.brost@intel.com, her0gyugyu@gmail.com Subject: [PATCH v16 18/42] dept: track timeout waits separately with a new Kconfig Date: Mon, 19 May 2025 18:18:02 +0900 Message-Id: <20250519091826.19752-19-byungchul@sk.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20250519091826.19752-1-byungchul@sk.com> References: <20250519091826.19752-1-byungchul@sk.com> X-Brightmail-Tracker: H4sIAAAAAAAAAzWSfUxTZxSH97739t7bzs5rMXqn+zBN2LRGGQaX4zI3s+zjjVHjx5YlanSN 3K2NUExREDMSOiu6ljIkQQS0FDS1KVVZaxQdNQijHX4AzkqhoUirITJAMrCdILq1mP1z8uR3 znnOP4ejFH7JIk6r2y/qdeosJSOjZWNz6lbk/KPSfGC6S0Hs6TEaTl10MdB9oQGB65IBw3D7 VxCMjyJ4fqeLgsqKbgR1kTAFl3wDCLyOnxi49+gNCMTGGeioMDNw+MxFBu6OzGDoP1GOocG9 ER7Yh2i4VVaPoXKYgZrKwzhRHmOYsjtZsBelQtRRzcJMJB06Bnok4A0thyprPwPN3g4afE1R DPeunWJgwPWvBG75/qAhXroYuo9bJHD+ST0DI3E7BfbYOAt/ttgw+GwLoNGYEBZPvpSA39KC ofjsrxgCfb8huH5sEIPb1cNAW2wUg8ddQcH0uXYE0dIxFo6UTLFQYyhFYD5yggZj/2p4/ixx +fTTdDDUNtJw/kUPWreWuKwuRNpGxyli9OST6dh9hnjjNprcrBfI1eowS4zXQyyxuQ8Qj0NF zjQPY1I3EZMQt/NnhrgnylliGgtg8qSzk9381nbZx5liljZP1Kd98p1Mc9YTpvaZ9xwMnSvH Rci/yYQ4TuAzhDabxoSks1g8YERJZvj3hd7eKSrJ8/klgscyJDEhGUfxPa8LwdN9s0Mp/FbB 2hRmkkzzqYIj4sdJlvMfCsbaFvRK+q7Q0NgyK5Im8pC5bTZX8KuFQIOVTkoF/hepYO0aYl8t vCnccPTSZUhuQ685kUKry8tWa7MyVmoKdNqDK/fkZLtR4r3shTM7mtBE97ZWxHNIOUfe6F2m UUjUebkF2a1I4CjlfLnTs1SjkGeqCw6J+pzd+gNZYm4rWszRyoXyVfH8TAX/g3q/uFcU94n6 /7uYky4qQimffhH4/nYtSb3SsUEatdSnBJd9O/fv8JpBQ/WLB7agZ4H+5NvGaJlnZl0k40Jh pCS45fM8+uiKXZc3+B7hkyHZ707tl7qUrx+rsv9aMvhs066PqmrMnXOPrt+s6upKC7X7Pzsu /Jje12kpmy4ZyXmPmjRM7nxY5U9rLnqncMj/Tf48JZ2rUaerKH2u+j+pjWXuWgMAAA== X-Brightmail-Tracker: H4sIAAAAAAAAAzWSa0iTYRTHfZ736mrxsqTeiqhGka2LSSkHsgv1oQe7EEUEFdSqlzZyszaz DAJtr8s0rQQVr02NKXPV2vpgl8VSctlNa8vK1NJCkiyhnOWlyzT6cvjx/x9+58vhKZWDmcnr jSmSyahNUrMKWrF1lWWp8YdGt3y0QQOhwSwayq47WWi9VofAeTMDQ9+DjfBqqB/B6NMWCooK WhFUdndScLOpC4G39gwLgY9TIBgaYKG5IIcFS/V1Fp5/HsPQUZiPoc69Bd7Ze2l4fLEKQ1Ef C6VFFhwenzAM2x0c2NMXQE9tCQdj3bHQ3NXGQGN5MwPe9sVQXNHBwl1vMw1N9T0YArfLWOhy /mHgcdNDGobyZkHrpVwGrn6tYuHzkJ0Ce2iAgxc+G4Ym2zRwyWGr9ftvBvy5PgzWKzcwBN/c QXAv6z0Gt7ONhcZQPwaPu4CCkZoHCHryvnCQeX6Yg9KMPAQ5mYU0yB1xMPozfLl8MBYyLrto uPqrDa1bQ5wVTkQa+wcoIntOkJHQS5Z4h2w0eVQlklslnRyR77VzxOY+Tjy1GlJ9tw+Tym8h hrgd51ji/pbPkewvQUy+PnvGbZu9W5FwSErSp0qmmDX7Fbornk7qaM7Bk+01+Tgd+bdmo0he FFaK1i4ZjTMrLBRfvx6mxjlKmCt6cnuZbKTgKaFtkviq/M3E0lRhu1hR38mOMy0sEGu7/Xic lUK8KF/2oX/SOWKdyzchigzn7TmNE7lKiBODdRX0RaSwoQgHitIbUw1afVLcMvMRXZpRf3LZ wWSDG4U/yH567FI9GgxsbEACj9STlS7vIp2K0aaa0wwNSOQpdZTS4YnWqZSHtGmnJFPyPtPx JMncgGbxtHq6MnGXtF8lHNamSEck6ahk+t9iPnJmOsKh+T9/H14SQSV37Jm3qT9w/4naKFgu eGMCttU18WeR357gX7dj/YrgZoel3uxfmDZ1btEBQ/Va8rFlOD3gaPkzY+eInCnrnprKbvn2 mpevYFoTPhw7bYjWPI+9EH17g1Uz9lZt6nUVxwzkb68yFCTyq+SIlKyHpZ/0KzOt8S1q2qzT xmook1n7F2kEE989AwAA X-CFilter-Loop: Reflected Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Waits with valid timeouts don't actually cause deadlocks. However, dept has been reporting the cases as well because it's worth informing the circular dependency for some cases where, for example, timeout is used to avoid a deadlock. However, yes, there are also a lot of, even more, cases where timeout is used for its clear purpose and meant to be expired. Report these as an information rather than warning DEADLOCK. Plus, introduce CONFIG_DEPT_AGGRESSIVE_TIMEOUT_WAIT Kconfig to make it optional so that any reports involving waits with timeouts can be turned on/off depending on the purpose. Signed-off-by: Byungchul Park --- include/linux/dept.h | 13 +++++--- include/linux/dept_ldt.h | 6 ++-- include/linux/dept_sdt.h | 13 +++++--- include/linux/sched.h | 2 ++ kernel/dependency/dept.c | 66 ++++++++++++++++++++++++++++++++++------ lib/Kconfig.debug | 10 ++++++ 6 files changed, 89 insertions(+), 21 deletions(-) diff --git a/include/linux/dept.h b/include/linux/dept.h index cb1b1beea077..49f457390521 100644 --- a/include/linux/dept.h +++ b/include/linux/dept.h @@ -271,6 +271,11 @@ struct dept_wait { * whether this wait is for commit in scheduler */ bool sched_sleep; + + /* + * whether a timeout is set + */ + bool timeout; }; }; }; @@ -377,8 +382,8 @@ extern void dept_free_range(void *start, unsigned int s= z); extern void dept_map_init(struct dept_map *m, struct dept_key *k, int sub_= u, const char *n); extern void dept_map_reinit(struct dept_map *m, struct dept_key *k, int su= b_u, const char *n); extern void dept_map_copy(struct dept_map *to, struct dept_map *from); -extern void dept_wait(struct dept_map *m, unsigned long w_f, unsigned long= ip, const char *w_fn, int sub_l); -extern void dept_stage_wait(struct dept_map *m, struct dept_key *k, unsign= ed long ip, const char *w_fn); +extern void dept_wait(struct dept_map *m, unsigned long w_f, unsigned long= ip, const char *w_fn, int sub_l, long timeout); +extern void dept_stage_wait(struct dept_map *m, struct dept_key *k, unsign= ed long ip, const char *w_fn, long timeout); extern void dept_request_event_wait_commit(void); extern void dept_clean_stage(void); extern void dept_ttwu_stage_wait(struct task_struct *t, unsigned long ip); @@ -425,8 +430,8 @@ struct dept_map { }; #define dept_map_init(m, k, su, n) do { (void)(n); (void)(k); } while (0) #define dept_map_reinit(m, k, su, n) do { (void)(n); (void)(k); } while = (0) #define dept_map_copy(t, f) do { } while (0) -#define dept_wait(m, w_f, ip, w_fn, sl) do { (void)(w_fn); } while (0) -#define dept_stage_wait(m, k, ip, w_fn) do { (void)(k); (void)(w_fn); } = while (0) +#define dept_wait(m, w_f, ip, w_fn, sl, t) do { (void)(w_fn); } while (0) +#define dept_stage_wait(m, k, ip, w_fn, t) do { (void)(k); (void)(w_fn); = } while (0) #define dept_request_event_wait_commit() do { } while (0) #define dept_clean_stage() do { } while (0) #define dept_ttwu_stage_wait(t, ip) do { } while (0) diff --git a/include/linux/dept_ldt.h b/include/linux/dept_ldt.h index 8047d0a531f1..730af2517ecd 100644 --- a/include/linux/dept_ldt.h +++ b/include/linux/dept_ldt.h @@ -28,7 +28,7 @@ else if (t) \ dept_ecxt_enter(m, LDT_EVT_L, i, "trylock", "unlock", sl);\ else { \ - dept_wait(m, LDT_EVT_L, i, "lock", sl); \ + dept_wait(m, LDT_EVT_L, i, "lock", sl, false); \ dept_ecxt_enter(m, LDT_EVT_L, i, "lock", "unlock", sl);\ } \ } while (0) @@ -40,7 +40,7 @@ else if (t) \ dept_ecxt_enter(m, LDT_EVT_R, i, "read_trylock", "read_unlock", sl);\ else { \ - dept_wait(m, q ? LDT_EVT_RW : LDT_EVT_W, i, "read_lock", sl);\ + dept_wait(m, q ? LDT_EVT_RW : LDT_EVT_W, i, "read_lock", sl, false);\ dept_ecxt_enter(m, LDT_EVT_R, i, "read_lock", "read_unlock", sl);\ } \ } while (0) @@ -52,7 +52,7 @@ else if (t) \ dept_ecxt_enter(m, LDT_EVT_W, i, "write_trylock", "write_unlock", sl);\ else { \ - dept_wait(m, LDT_EVT_RW, i, "write_lock", sl); \ + dept_wait(m, LDT_EVT_RW, i, "write_lock", sl, false);\ dept_ecxt_enter(m, LDT_EVT_W, i, "write_lock", "write_unlock", sl);\ } \ } while (0) diff --git a/include/linux/dept_sdt.h b/include/linux/dept_sdt.h index 0535f763b21b..14917df0cc30 100644 --- a/include/linux/dept_sdt.h +++ b/include/linux/dept_sdt.h @@ -23,11 +23,12 @@ =20 #define sdt_map_init_key(m, k) dept_map_init(m, k, 0, #m) =20 -#define sdt_wait(m) \ +#define sdt_wait_timeout(m, t) \ do { \ dept_request_event(m); \ - dept_wait(m, 1UL, _THIS_IP_, __func__, 0); \ + dept_wait(m, 1UL, _THIS_IP_, __func__, 0, t); \ } while (0) +#define sdt_wait(m) sdt_wait_timeout(m, -1L) =20 /* * sdt_might_sleep() and its family will be committed in __schedule() @@ -38,13 +39,13 @@ /* * Use the code location as the class key if an explicit map is not used. */ -#define sdt_might_sleep_start(m) \ +#define sdt_might_sleep_start_timeout(m, t) \ do { \ struct dept_map *__m =3D m; \ static struct dept_key __key; \ - dept_stage_wait(__m, __m ? NULL : &__key, _THIS_IP_, __func__);\ + dept_stage_wait(__m, __m ? NULL : &__key, _THIS_IP_, __func__, t);\ } while (0) - +#define sdt_might_sleep_start(m) sdt_might_sleep_start_timeout(m, -1L) #define sdt_might_sleep_end() dept_clean_stage() =20 #define sdt_ecxt_enter(m) dept_ecxt_enter(m, 1UL, _THIS_IP_, "start", "ev= ent", 0) @@ -54,7 +55,9 @@ #else /* !CONFIG_DEPT */ #define sdt_map_init(m) do { } while (0) #define sdt_map_init_key(m, k) do { (void)(k); } while (0) +#define sdt_wait_timeout(m, t) do { } while (0) #define sdt_wait(m) do { } while (0) +#define sdt_might_sleep_start_timeout(m, t) do { } while (0) #define sdt_might_sleep_start(m) do { } while (0) #define sdt_might_sleep_end() do { } while (0) #define sdt_ecxt_enter(m) do { } while (0) diff --git a/include/linux/sched.h b/include/linux/sched.h index 802fca4d99b3..541ddacdc3d0 100644 --- a/include/linux/sched.h +++ b/include/linux/sched.h @@ -864,6 +864,7 @@ struct dept_task { bool stage_sched_map; const char *stage_w_fn; unsigned long stage_ip; + bool stage_timeout; arch_spinlock_t stage_lock; =20 /* @@ -904,6 +905,7 @@ struct dept_task { .stage_sched_map =3D false, \ .stage_w_fn =3D NULL, \ .stage_ip =3D 0UL, \ + .stage_timeout =3D false, \ .stage_lock =3D (arch_spinlock_t)__ARCH_SPIN_LOCK_UNLOCKED,\ .missing_ecxt =3D 0, \ .hardirqs_enabled =3D false, \ diff --git a/kernel/dependency/dept.c b/kernel/dependency/dept.c index a2384f3148c5..a6abba8f3a2c 100644 --- a/kernel/dependency/dept.c +++ b/kernel/dependency/dept.c @@ -757,6 +757,8 @@ static void print_diagram(struct dept_dep *d) if (!irqf) { print_spc(spc, "[S] %s(%s:%d)\n", c_fn, fc_n, fc->sub_id); print_spc(spc, "[W] %s(%s:%d)\n", w_fn, tc_n, tc->sub_id); + if (w->timeout) + print_spc(spc, "--------------- >8 timeout ---------------\n"); print_spc(spc, "[E] %s(%s:%d)\n", e_fn, fc_n, fc->sub_id); } } @@ -810,6 +812,24 @@ static void print_dep(struct dept_dep *d) =20 static void save_current_stack(int skip); =20 +static bool is_timeout_wait_circle(struct dept_class *c) +{ + struct dept_class *fc =3D c->bfs_parent; + struct dept_class *tc =3D c; + + do { + struct dept_dep *d =3D lookup_dep(fc, tc); + + if (d->wait->timeout) + return true; + + tc =3D fc; + fc =3D fc->bfs_parent; + } while (tc !=3D c); + + return false; +} + /* * Print all classes in a circle. */ @@ -832,10 +852,14 @@ static void print_circle(struct dept_class *c) pr_warn("summary\n"); pr_warn("---------------------------------------------------\n"); =20 - if (fc =3D=3D tc) + if (is_timeout_wait_circle(c)) { + pr_warn("NOT A DEADLOCK BUT A CIRCULAR DEPENDENCY\n"); + pr_warn("CHECK IF THE TIMEOUT IS INTENDED\n\n"); + } else if (fc =3D=3D tc) { pr_warn("*** AA DEADLOCK ***\n\n"); - else + } else { pr_warn("*** DEADLOCK ***\n\n"); + } =20 i =3D 0; do { @@ -1579,7 +1603,8 @@ static int next_wgen(void) } =20 static void add_wait(struct dept_class *c, unsigned long ip, - const char *w_fn, int sub_l, bool sched_sleep) + const char *w_fn, int sub_l, bool sched_sleep, + bool timeout) { struct dept_task *dt =3D dept_task(); struct dept_wait *w; @@ -1599,6 +1624,7 @@ static void add_wait(struct dept_class *c, unsigned l= ong ip, w->wait_fn =3D w_fn; w->wait_stack =3D get_current_stack(); w->sched_sleep =3D sched_sleep; + w->timeout =3D timeout; =20 cxt =3D cur_cxt(); if (cxt =3D=3D DEPT_CXT_HIRQ || cxt =3D=3D DEPT_CXT_SIRQ) @@ -2297,7 +2323,7 @@ static struct dept_class *check_new_class(struct dept= _key *local, */ static void __dept_wait(struct dept_map *m, unsigned long w_f, unsigned long ip, const char *w_fn, int sub_l, - bool sched_sleep, bool sched_map) + bool sched_sleep, bool sched_map, bool timeout) { int e; =20 @@ -2320,7 +2346,7 @@ static void __dept_wait(struct dept_map *m, unsigned = long w_f, if (!c) continue; =20 - add_wait(c, ip, w_fn, sub_l, sched_sleep); + add_wait(c, ip, w_fn, sub_l, sched_sleep, timeout); } } =20 @@ -2355,14 +2381,23 @@ static void __dept_event(struct dept_map *m, struct= dept_map *real_m, } =20 void dept_wait(struct dept_map *m, unsigned long w_f, - unsigned long ip, const char *w_fn, int sub_l) + unsigned long ip, const char *w_fn, int sub_l, + long timeoutval) { struct dept_task *dt =3D dept_task(); unsigned long flags; + bool timeout; =20 if (unlikely(!dept_working())) return; =20 + timeout =3D timeoutval > 0 && timeoutval < MAX_SCHEDULE_TIMEOUT; + +#if !defined(CONFIG_DEPT_AGGRESSIVE_TIMEOUT_WAIT) + if (timeout) + return; +#endif + if (dt->recursive) return; =20 @@ -2371,21 +2406,30 @@ void dept_wait(struct dept_map *m, unsigned long w_= f, =20 flags =3D dept_enter(); =20 - __dept_wait(m, w_f, ip, w_fn, sub_l, false, false); + __dept_wait(m, w_f, ip, w_fn, sub_l, false, false, timeout); =20 dept_exit(flags); } EXPORT_SYMBOL_GPL(dept_wait); =20 void dept_stage_wait(struct dept_map *m, struct dept_key *k, - unsigned long ip, const char *w_fn) + unsigned long ip, const char *w_fn, + long timeoutval) { struct dept_task *dt =3D dept_task(); unsigned long flags; + bool timeout; =20 if (unlikely(!dept_working())) return; =20 + timeout =3D timeoutval > 0 && timeoutval < MAX_SCHEDULE_TIMEOUT; + +#if !defined(CONFIG_DEPT_AGGRESSIVE_TIMEOUT_WAIT) + if (timeout) + return; +#endif + if (m && m->nocheck) return; =20 @@ -2434,6 +2478,7 @@ void dept_stage_wait(struct dept_map *m, struct dept_= key *k, =20 dt->stage_w_fn =3D w_fn; dt->stage_ip =3D ip; + dt->stage_timeout =3D timeout; arch_spin_unlock(&dt->stage_lock); exit: dept_exit_recursive(flags); @@ -2447,6 +2492,7 @@ static void __dept_clean_stage(struct dept_task *dt) dt->stage_sched_map =3D false; dt->stage_w_fn =3D NULL; dt->stage_ip =3D 0UL; + dt->stage_timeout =3D false; } =20 void dept_clean_stage(void) @@ -2479,6 +2525,7 @@ void dept_request_event_wait_commit(void) unsigned long ip; const char *w_fn; bool sched_map; + bool timeout; =20 if (unlikely(!dept_working())) return; @@ -2505,12 +2552,13 @@ void dept_request_event_wait_commit(void) w_fn =3D dt->stage_w_fn; ip =3D dt->stage_ip; sched_map =3D dt->stage_sched_map; + timeout =3D dt->stage_timeout; =20 wg =3D next_wgen(); WRITE_ONCE(dt->stage_m.wgen, wg); arch_spin_unlock(&dt->stage_lock); =20 - __dept_wait(&dt->stage_m, 1UL, ip, w_fn, 0, true, sched_map); + __dept_wait(&dt->stage_m, 1UL, ip, w_fn, 0, true, sched_map, timeout); exit: dept_exit(flags); } diff --git a/lib/Kconfig.debug b/lib/Kconfig.debug index 195c11b92bcf..ec840c672846 100644 --- a/lib/Kconfig.debug +++ b/lib/Kconfig.debug @@ -1394,6 +1394,16 @@ config DEPT noting, to mitigate the impact by the false positives, multi reporting has been supported. =20 +config DEPT_AGGRESSIVE_TIMEOUT_WAIT + bool "Aggressively track even timeout waits" + depends on DEPT + default n + help + Timeout wait doesn't contribute to a deadlock. However, + informing a circular dependency might be helpful for cases + that timeout is used to avoid a deadlock. Say N if you'd like + to avoid verbose reports. + config LOCK_DEBUGGING_SUPPORT bool depends on TRACE_IRQFLAGS_SUPPORT && STACKTRACE_SUPPORT && LOCKDEP_SUPPORT --=20 2.17.1 From nobody Fri Dec 19 15:51:27 2025 Received: from invmail4.hynix.com (exvmail4.hynix.com [166.125.252.92]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 02EC3274FE5; Mon, 19 May 2025 09:18:54 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=166.125.252.92 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747646338; cv=none; b=eVIWCny6MceHJqziLfLiTOdC5pJKHDZCjEXKGoqI96tplV03U2hYQmNpofhYSeRBk3PlS8VKLayLDNEx9DvWzzUHfcB7G1iRJmNTaAvTtncgKeoZLkecd6D4U0BQiCxt2LV9dtXV6EldUnVTdHJ324uTYKEAqxuQPLyZXYG1vpE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747646338; c=relaxed/simple; bh=TSPxdh3H9mjhA/wDLLlJUz6YZTY7hYg5ulgpaHBJJSM=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References; b=sDQySTOpztmCCAhbkOo8TgjHH6KuwYtNmz/LCdfcHxFHIhEMMGcRNfOUL0FfKfRILq6a/MazSejvtgBGo4f8Tk0VW59XKVzk6Chf3AD4d+wupFsfiqI/dqDyr0TnjQOh5BuAlXluJaGTGaY1DYyzRN9R9/Bk/UAl96RbT4c/1V8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com; spf=pass smtp.mailfrom=sk.com; arc=none smtp.client-ip=166.125.252.92 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=sk.com X-AuditID: a67dfc5b-681ff7000002311f-04-682af76e7825 From: Byungchul Park To: linux-kernel@vger.kernel.org Cc: kernel_team@skhynix.com, torvalds@linux-foundation.org, damien.lemoal@opensource.wdc.com, linux-ide@vger.kernel.org, adilger.kernel@dilger.ca, linux-ext4@vger.kernel.org, mingo@redhat.com, peterz@infradead.org, will@kernel.org, tglx@linutronix.de, rostedt@goodmis.org, joel@joelfernandes.org, sashal@kernel.org, daniel.vetter@ffwll.ch, duyuyang@gmail.com, johannes.berg@intel.com, tj@kernel.org, tytso@mit.edu, willy@infradead.org, david@fromorbit.com, amir73il@gmail.com, gregkh@linuxfoundation.org, kernel-team@lge.com, linux-mm@kvack.org, akpm@linux-foundation.org, mhocko@kernel.org, minchan@kernel.org, hannes@cmpxchg.org, vdavydov.dev@gmail.com, sj@kernel.org, jglisse@redhat.com, dennis@kernel.org, cl@linux.com, penberg@kernel.org, rientjes@google.com, vbabka@suse.cz, ngupta@vflare.org, linux-block@vger.kernel.org, josef@toxicpanda.com, linux-fsdevel@vger.kernel.org, jack@suse.cz, jlayton@kernel.org, dan.j.williams@intel.com, hch@infradead.org, djwong@kernel.org, dri-devel@lists.freedesktop.org, rodrigosiqueiramelo@gmail.com, melissa.srw@gmail.com, hamohammed.sa@gmail.com, harry.yoo@oracle.com, chris.p.wilson@intel.com, gwan-gyeong.mun@intel.com, max.byungchul.park@gmail.com, boqun.feng@gmail.com, longman@redhat.com, yskelg@gmail.com, yunseong.kim@ericsson.com, yeoreum.yun@arm.com, netdev@vger.kernel.org, matthew.brost@intel.com, her0gyugyu@gmail.com Subject: [PATCH v16 19/42] dept: apply timeout consideration to wait_for_completion()/complete() Date: Mon, 19 May 2025 18:18:03 +0900 Message-Id: <20250519091826.19752-20-byungchul@sk.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20250519091826.19752-1-byungchul@sk.com> References: <20250519091826.19752-1-byungchul@sk.com> X-Brightmail-Tracker: H4sIAAAAAAAAAzWSa0hTcRjG+//P2Tln08VhRZ20iw3ENLoYGe+HkqAPHaIoib7YB1t5aiMv MS+1QHKlkVfSUslqTY01dOY6M7LLzDRXdtVcS0UtrUzxMrC20tTajL68PDwPz+/98jCEwiEJ YjRJqYI2SZWgpGSkbDywYl3yzwj1xqnWQPD8uEDCtToLBe23axBY6vUYRlp3wgfvGILfr98S UFbSjqBioI+Aekc/Arv5LAWdXxaC0+OmoK0kj4JzVXUUdIzOYOgtLcZQI+6Bj6YhEl5erMRQ NkLB1bJz2HeGMUyZqmkwZYbCoLmchpmBSGjrd0nA3rMWrhh6KXhkbyPB0TCIofPBNQr6LX8k 8NLxnARvYTC0FxVIoHaikoJRr4kAk8dNw7smIwaHcQlYs3zA89/nJPCsoAnD+Zt3MDi7HyJo vPAJg2hxUdDiGcNgE0sImL7VimCwcJyG7PwpGq7qCxHkZZeSkNUbBb9/+T5f/xEJ+htWEmpn XWj7Nt5isCC+ZcxN8Fm2k/y05z3F271Gkn9RyfH3y/toPquxh+aNYhpvM0fwVY9GMF8x6ZHw YnUOxYuTxTSfO+7E/MSbN/S+5bGyrfFCgiZd0G6IPiRTW95doU8YZKdsOR04E31ichHDcOxm zjqgzEXSeTk6JEr8mmLDuK6uKcKvF7MhnK1gyOfLGIJ1BXAfrncjf7CIPcQZOovmCyQbyokd pdjPlLNbuIFmxT/mKq7G2jTPkfrsnryW+aqCjeKcNQbSz+TYKiknPniG/hWWcU/MXeRFJDei BdVIoUlKT1RpEjavV+uSNKfWH0lOFJFvXaaMmYMNaLJ9fzNiGaQMlFvt4WqFRJWeoktsRhxD KBfLq21r1Ap5vEp3WtAmx2nTEoSUZhTMkMql8k3ek/EK9pgqVTguCCcE7f8UM9KgTBQbvvqw d9eXrWnSHTHufPL157haz962zOBvsWxeE6rXBRTPXlas3DGR0U3VTR8N2vQiZHtMoSb12N0M PWbXxnUX6VRbYqJ1LUvw0+ez5FnnmbnaEfPchu+hwzeNjfl9rpBoMsx8qVGm/xqX8/iAbcHu M/vu4HuvJtThAdnqFe5yJZmiVkVGENoU1V8NdDtPWQMAAA== X-Brightmail-Tracker: H4sIAAAAAAAAAzWSe0xTZxjG/b5zpVpzUpkeNWbSBC8Ynajo6yXGRRNOjBBMNCaEZFQ9sQ1Q tFWUGZMi4LgIERQIIF3B2ZW2ju4UIypV0kYU3QTkoiCixQVFUKLSMhDFgvGfN0+eJ8/v/edh CUU1tYDVaI+KOq0qUUnLSFn05vSV2tEw9erqp2vAN5JFwsUaOw0tf9kQ2GvTMAzciYTH/iEE n/5tJqCkqAVBpfcZAbWNvQhcltM0tP03G9p9wzQ0FeXSkH6phobWwQkMPcWFGGxSFDw395Pw 4FwVhpIBGspL0nHgvMYwZrYyYDaEQp+ljIEJbzg09XZS4KloosDVvQJKjT001LuaSGis68PQ duMiDb32SQoeNN4jwZ+/EFoK8ii48q6KhkG/mQCzb5iBRw0mDI2mueDICFDPfPxCwd28Bgxn /vgbQ3vXTQS3sl5gkOydNHh8QxicUhEB43/eQdCX/5aBzLNjDJSn5SPIzSwmIaMnAj79H/hc MRIOab87SLjyuRNt2yrYjXYkeIaGCSHDeVwY93XQgstvIoX7VbxwvewZI2Tc6mYEk3RMcFrC hEv1A1io/OCjBMmaTQvSh0JGyHnbjoV3Dx8yMYtiZVsOiomaFFH309Z4mdr+qJQ5bJSdcGa3 YgN6weagIJbn1vGD/RI1pWluKf/kyRgxpYO5xbwzrz/gy1iC65zJP67oQlPBHC6eN7YVTBdI LpSXWotxDmJZObee97oV35g/8jZHwzQnKGB353qmqwougm+3GclzSGZCM6woWKNNSVJpEiNW 6RPUqVrNiVUHkpMkFBiQ+dREQR0aaYt0I45Fyllyh2u5WkGpUvSpSW7Es4QyWG51LlMr5AdV qb+KuuRfdMcSRb0bLWRJ5Tz5zn1ivII7pDoqJojiYVH3PcVs0AIDWrZpXf755MKaNxEXdrv2 xEVumNy7v2H/0sjcGPXaZsk72Xc1RRl9IXt+mOX9vgSP7zq+vKt0ZUz6yeNLro1vDvH2PG/2 72Dj9vycudcw0WXtuH3q5a5ZaT+8ii3ZlHXPMWpYgyps9eWvopZb6o6ExFbH7e5oHY0e2DjP zWypDd1+9rd/lKRerQoPI3R61VdrWUlHPAMAAA== X-CFilter-Loop: Reflected Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Now that CONFIG_DEPT_AGGRESSIVE_TIMEOUT_WAIT was introduced, apply the consideration to wait_for_completion()/complete(). Signed-off-by: Byungchul Park --- include/linux/completion.h | 4 ++-- kernel/sched/completion.c | 2 +- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/include/linux/completion.h b/include/linux/completion.h index bd2c207481d6..3200b741de28 100644 --- a/include/linux/completion.h +++ b/include/linux/completion.h @@ -41,9 +41,9 @@ do { \ */ #define init_completion_map(x, m) init_completion(x) =20 -static inline void complete_acquire(struct completion *x) +static inline void complete_acquire(struct completion *x, long timeout) { - sdt_might_sleep_start(&x->dmap); + sdt_might_sleep_start_timeout(&x->dmap, timeout); } =20 static inline void complete_release(struct completion *x) diff --git a/kernel/sched/completion.c b/kernel/sched/completion.c index 3561ab533dd4..499b1fee9dc1 100644 --- a/kernel/sched/completion.c +++ b/kernel/sched/completion.c @@ -110,7 +110,7 @@ __wait_for_common(struct completion *x, { might_sleep(); =20 - complete_acquire(x); + complete_acquire(x, timeout); =20 raw_spin_lock_irq(&x->wait.lock); timeout =3D do_wait_for_common(x, action, timeout, state); --=20 2.17.1 From nobody Fri Dec 19 15:51:27 2025 Received: from invmail4.hynix.com (exvmail4.hynix.com [166.125.252.92]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 1D2EF274FF6; Mon, 19 May 2025 09:18:54 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=166.125.252.92 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747646338; cv=none; b=pOdwFKKdVaCqraWOesCUqbm+F8Yl9GqBmy30JcJmrA9PSu4A8l9FEAZ9Nu7UA9Te2AlFb/4nac5oP3XaEq0MXrJxRo70F8rz51ReFE72B2zGR1/gZessBH7za36A/PWRQyE6m6ttIgzzC4WbxtXgnFWR6Nsh9XjwSMFO/lW9hbY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747646338; c=relaxed/simple; bh=9o9SI0ONdb1WbiIWXrCSb5nX53qir4UGX5Gt0IQgpYM=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References; b=Bp6MlfOh7En8dDbPJMbOGLQKRAjCOpgq8BgDck9TXPUNKrPKoGI1VpF3WRklP2PECvVxjI0PVYnAfkPL9cSvouV66xMCDtann4JfpgoNUMIb4C4S5vS2wzodYbgobT9gwbxqQTam8jHBKq3q/dHuAGaKjbvUWonOiCxmB/ZsAsg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com; spf=pass smtp.mailfrom=sk.com; arc=none smtp.client-ip=166.125.252.92 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=sk.com X-AuditID: a67dfc5b-681ff7000002311f-13-682af76ee140 From: Byungchul Park To: linux-kernel@vger.kernel.org Cc: kernel_team@skhynix.com, torvalds@linux-foundation.org, damien.lemoal@opensource.wdc.com, linux-ide@vger.kernel.org, adilger.kernel@dilger.ca, linux-ext4@vger.kernel.org, mingo@redhat.com, peterz@infradead.org, will@kernel.org, tglx@linutronix.de, rostedt@goodmis.org, joel@joelfernandes.org, sashal@kernel.org, daniel.vetter@ffwll.ch, duyuyang@gmail.com, johannes.berg@intel.com, tj@kernel.org, tytso@mit.edu, willy@infradead.org, david@fromorbit.com, amir73il@gmail.com, gregkh@linuxfoundation.org, kernel-team@lge.com, linux-mm@kvack.org, akpm@linux-foundation.org, mhocko@kernel.org, minchan@kernel.org, hannes@cmpxchg.org, vdavydov.dev@gmail.com, sj@kernel.org, jglisse@redhat.com, dennis@kernel.org, cl@linux.com, penberg@kernel.org, rientjes@google.com, vbabka@suse.cz, ngupta@vflare.org, linux-block@vger.kernel.org, josef@toxicpanda.com, linux-fsdevel@vger.kernel.org, jack@suse.cz, jlayton@kernel.org, dan.j.williams@intel.com, hch@infradead.org, djwong@kernel.org, dri-devel@lists.freedesktop.org, rodrigosiqueiramelo@gmail.com, melissa.srw@gmail.com, hamohammed.sa@gmail.com, harry.yoo@oracle.com, chris.p.wilson@intel.com, gwan-gyeong.mun@intel.com, max.byungchul.park@gmail.com, boqun.feng@gmail.com, longman@redhat.com, yskelg@gmail.com, yunseong.kim@ericsson.com, yeoreum.yun@arm.com, netdev@vger.kernel.org, matthew.brost@intel.com, her0gyugyu@gmail.com Subject: [PATCH v16 20/42] dept: apply timeout consideration to swait Date: Mon, 19 May 2025 18:18:04 +0900 Message-Id: <20250519091826.19752-21-byungchul@sk.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20250519091826.19752-1-byungchul@sk.com> References: <20250519091826.19752-1-byungchul@sk.com> X-Brightmail-Tracker: H4sIAAAAAAAAAzWSbUxTZxTHfe7Lc28rnddi4CpNNE2MivMFg+zMOLNkWbgxITPZh2Uao1Vu bGML2ipQExc6kAgMAkZgWmXlJYXRKvXWRHwpQRhFxJeCFZAASkUckUoGtIqiW+u2Lye/nPPP 75wPhyWVnfQKVpdxTDRmaPRqLKfkwZiaDZlvErWb5/I4CM2dpuBCsxOD77IDgfOqhYDJzlQY CE8heH//IQlVFT4ENWMjJFz1jiLwNP6M4dH4Z+APTWPorijGkFfXjKH31QIBw5VnCHBIafDU PkFBT1ktAVWTGKxVeUSk/EnAvL2JAXvuagg0nmdgYSwJukf7afAMrYdz1cMYbnm6KfC2BAh4 dOMChlHn3zT0eO9QEC5NAF95CQ2XXtdieBW2k2APTTPQ12YjwGuLA1d+RFgw+5GGrpI2Agrq rxDgf3ITQevpZwRIzn4MHaEpAtxSBQnvGjoRBEqDDJz6ZZ4Bq6UUQfGpSgryh7fC+7eRzRfn ksDym4uCSx/60ddfCc5qJxI6pqZJId+dLbwLPcaCJ2yjhLu1vHD9/Agj5LcOMYJNOi64GxOF uluThFAzE6IFqakQC9LMGUYoCvoJ4fWDB8wu1W759nRRr8sSjZt27Jdrm8ct6MgszqnqrKdz 0V26CMlYnkvmC6xBVITYT+x3EdE25tbwg4PzZJSXcat4d8lEJC5nSa5/MT9w8Ukkz7Cx3Lf8 c1k0QnGr+fo/2qgoK7gU3jVjxf/aV/IOV9snjSzSHyruQFFWclt5v6Oaiip5rkHGnz079985 y/nbjYNUGVLY0KImpNRlZBk0On3yRq05Q5ez8WCmQUKR57KfXNjTgmZ837cjjkXqGIXLs06r pDVZJrOhHfEsqV6maHKv1SoV6RrzCdGYuc94XC+a2lECS6njFVvC2elK7pDmmHhYFI+Ixv+n BCtbkYuSc75YyqYYDTv0DXu7DgZ+j3uBzT+WHdriSDzQG6zo3i2V31vOdk1lX/uBX7lq4LuE Bf1fi/zpcuXRK4V9utsHdj7UmdKu/5TiU1lnVS2xZSpV/JepbxaXq5eqDalpR8fl2pc3X1ae y4uRvjGz7b/O74xptSTtKvR+rEvVxH0e2LZETZm0mqRE0mjS/ANV28cNWAMAAA== X-Brightmail-Tracker: H4sIAAAAAAAAAzWSb0xTZxjF97733vfeVmtuOtxucMalic7hVEiGe5IZoh+mNyQsLlkkkWyz GzdrFSq5pSBLTOgAx4ogyICMCit/UkhbRteyjaFlhEpndTqUrgoBNpkjEhASoJUKbmtN9uXJ L885OefL4Sh1F5PM6Q2FkmzQ5mmIkla++3bZXsOTFF3qVz8DRFYrabjc6yIw+q0TgavPjGFu 5Cjciy4gWL/1GwVNDaMI2h5MUdAXmEbg6/6cwNjDLRCKLBEINlQRKOvoJXBnfgPDZOMlDE5P Fvxhn6XhZm07hqY5AtamMhw/jzDE7A4W7KU7Yaa7mYWNB2kQnA4z4G8JMuCb2ANft04SuOoL 0hDon8EwNnCZwLTrXwZuBq7TEK3ZBqN11Qz0LLYTmI/aKbBHlli4O2TDELC9BO7yeOr5lX8Y +KV6CMP5zu8whMavIBis/BODxxUm4I8sYPB6Gih42jWCYKbmMQsVF2IsWM01CKoqGmkon0yH 9bV4c8tqGpi/cdPQ8yyMDmWIrlYXEv0LS5RY7i0Wn0Z+J6IvaqPFG+2C+FPzFCuWD06wos1j Er3dKWLH1Tksti1HGNHj+JKInuVLrGh5HMLi4u3b7LHtJ5QHc6U8fZEk7884qdT1PjSjghVy tmmkkylFNxgL4jiBf1MIubEFKTjCvybcvx+jEpzEvyp4q2fjFiVH8eFNwr2WcWRBLPci/47w lyJhofmdQue1ITrBKv6A4F62kgQL/A7B6R56HqOI/yeq/CjBaj5dCDlb6VqktKEXHChJbyjK 1+rz0vcZT+tKDPqz+z45k+9B8fnYz23U9aPVsaPDiOeQZrPK7Xtdp2a0RcaS/GEkcJQmSeXw 7tapVbnaks8k+cxHsilPMg6jbRyteVmVmS2dVPOfagul05JUIMn/q5hTJJci/OGzjqzd9QPH 1xsPHiner5yJOQPXwuiDt9ambIcs9a+0hbcnn4r1H+aaTVl7qSv6jPeO/ejNVtcfXh3M3azX vLF48decXRVb+hRb/cG/swuP1PbsmS8wk8r22YJiayxzWZNzyjq89oVM3fqhNfPj8dTU6TqT PPD+iul7R9dgVD6goY06bVoKJRu1/wE2pECfOgMAAA== X-CFilter-Loop: Reflected Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Now that CONFIG_DEPT_AGGRESSIVE_TIMEOUT_WAIT was introduced, apply the consideration to swait, assuming an input 'ret' in ___swait_event() macro is used as a timeout value. Signed-off-by: Byungchul Park --- include/linux/swait.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/include/linux/swait.h b/include/linux/swait.h index 277ac74f61c3..233acdf55e9b 100644 --- a/include/linux/swait.h +++ b/include/linux/swait.h @@ -162,7 +162,7 @@ extern void finish_swait(struct swait_queue_head *q, st= ruct swait_queue *wait); struct swait_queue __wait; \ long __ret =3D ret; \ \ - sdt_might_sleep_start(NULL); \ + sdt_might_sleep_start_timeout(NULL, __ret); \ INIT_LIST_HEAD(&__wait.task_list); \ for (;;) { \ long __int =3D prepare_to_swait_event(&wq, &__wait, state);\ --=20 2.17.1 From nobody Fri Dec 19 15:51:27 2025 Received: from invmail4.hynix.com (exvmail4.skhynix.com [166.125.252.92]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 71E2B275113; Mon, 19 May 2025 09:18:55 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=166.125.252.92 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747646339; cv=none; b=byVzmAUClKI9oihWr6X0yO6cUQcEN/O4gQqc0w04CG7/f2F/VG61flAMVbAXEiG/PIklrm3Z+vggsBvGtHhwvmcTQ/pHGFsLGFkBUsVMRpAVREkSbRmD1xcexumM/eV5amIGLdKMBLPeY/SRdxsCIG9TBZHLpDxpRfJdhMor5x8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747646339; c=relaxed/simple; bh=G02jUxrc8GHX4FLfxPcv5jAlvBb9vcSmciPzIKTL9pw=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References; b=LdEYFWd1HACxfmUE6wcuQxStbg3IbiRUsbTZ2LqdPOsHE4/7DaToBuh6Xwvh5Xdwnju9920ybftNBHms6D+JXNG9kzlkENuU7WFCE8FOz0pQ4AVXknhkOi8t5D6EHkF521i9K9amQdNcXtPIsyY7FEIRrJVQkTKqmcyZ7eoN+kA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com; spf=pass smtp.mailfrom=sk.com; arc=none smtp.client-ip=166.125.252.92 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=sk.com X-AuditID: a67dfc5b-681ff7000002311f-22-682af76f0825 From: Byungchul Park To: linux-kernel@vger.kernel.org Cc: kernel_team@skhynix.com, torvalds@linux-foundation.org, damien.lemoal@opensource.wdc.com, linux-ide@vger.kernel.org, adilger.kernel@dilger.ca, linux-ext4@vger.kernel.org, mingo@redhat.com, peterz@infradead.org, will@kernel.org, tglx@linutronix.de, rostedt@goodmis.org, joel@joelfernandes.org, sashal@kernel.org, daniel.vetter@ffwll.ch, duyuyang@gmail.com, johannes.berg@intel.com, tj@kernel.org, tytso@mit.edu, willy@infradead.org, david@fromorbit.com, amir73il@gmail.com, gregkh@linuxfoundation.org, kernel-team@lge.com, linux-mm@kvack.org, akpm@linux-foundation.org, mhocko@kernel.org, minchan@kernel.org, hannes@cmpxchg.org, vdavydov.dev@gmail.com, sj@kernel.org, jglisse@redhat.com, dennis@kernel.org, cl@linux.com, penberg@kernel.org, rientjes@google.com, vbabka@suse.cz, ngupta@vflare.org, linux-block@vger.kernel.org, josef@toxicpanda.com, linux-fsdevel@vger.kernel.org, jack@suse.cz, jlayton@kernel.org, dan.j.williams@intel.com, hch@infradead.org, djwong@kernel.org, dri-devel@lists.freedesktop.org, rodrigosiqueiramelo@gmail.com, melissa.srw@gmail.com, hamohammed.sa@gmail.com, harry.yoo@oracle.com, chris.p.wilson@intel.com, gwan-gyeong.mun@intel.com, max.byungchul.park@gmail.com, boqun.feng@gmail.com, longman@redhat.com, yskelg@gmail.com, yunseong.kim@ericsson.com, yeoreum.yun@arm.com, netdev@vger.kernel.org, matthew.brost@intel.com, her0gyugyu@gmail.com Subject: [PATCH v16 21/42] dept: apply timeout consideration to waitqueue wait Date: Mon, 19 May 2025 18:18:05 +0900 Message-Id: <20250519091826.19752-22-byungchul@sk.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20250519091826.19752-1-byungchul@sk.com> References: <20250519091826.19752-1-byungchul@sk.com> X-Brightmail-Tracker: H4sIAAAAAAAAAzWSa0xTdxjG/Z97O6vHavAoJs5GgpToZIJ5XdSYGOMxy8wysw+bH7TKmW0s hZS7iQmX4gSECYpkCFiKKQ0to55SL4wql1hlbIiC5TKKQpSIFkmQFkGQFZxf3jx53ie/9/nw MrjcTa5nNLokQa9TaRWUlJCOL6/aGj+tVG+314WAf+o8AeX1Ngq6/rAisDVkYjB2/yD0BnwI PvzzCIfSki4EVcNeHBrcQwhcliwKul+sgB7/BAXtJfkUZFfXU/D4zRwGg1eKMbCK38Ez8ygB HRdNGJSOUXC1NBsLjlcYzJhraTBnhMGIpYyGueEoaB/ykOAaiITfKwcpaHK1E+C+PYJBd2M5 BUO2BRI63A8JCBSGQldRAQl1b00UvAmYcTD7J2h40mzEwG0MAbshCDz37iMJDwqaMTh3/QYG Pf1/Irh7/jkGos1DQZvfh4FDLMFhtuY+gpHCcRpyLszQcDWzEEF+zhUCDIMx8OF98HLFVBRk XrMTUDfvQfv28LZKG+LbfBM4b3Ck8rP+pxTvChgJ/i8Tx98p89K84e4AzRvFZN5hUfLVTWMY XzXpJ3mxNpfixclims8b78H4t52d9PcbfpbujhW0mhRB/9Xe41L1YLGBTHDSadaaLCoDGag8 JGE4NppranhGftY3/K+WfIoN5/r6ZvBFvYb9knMUjAYzUgZnPV9wvRX9KA8xzGr2MHdvZuui JNgw7vlI8mJcxu7kPOVZ6BNyI2e1Ny9hJEF/IL9tyZezMVyPtZJYRHLsbxKu9NfH/3dYx7VY +oiLSGZEy2qRXKNLiVNptNHb1Ok6Tdq2k/FxIgp+l/ns3NHbaLLrSCtiGaRYLrO7ItRyUpWS mB7XijgGV6yR1Tq2qOWyWFX6GUEff0yfrBUSW1EoQyjWyr4OpMbK2VOqJOG0ICQI+s9bjJGs z0C/EN/Ud3rBj4dNk9Jc2d/hLXTZbt38wo8mp+lExbvX09u1ktxL4kCE7QAjbrqlDP+hWVe0 yvko69D4+59kO+/d9Oa0v8yezZ/yKR4qI6ynI1eczN3hu5xnmS+qafy3/tBHnXNf6NHs16sz JkO8K+Wj3y6M9sd3MI3O/ZsjdqVuIBREoloVpcT1iar/APiNBupZAwAA X-Brightmail-Tracker: H4sIAAAAAAAAAzWSa0hTcRiH+5+7q8VhiR20ciy6GVmSxhvdP0SnIPFbEEUd6tCGc5NNVxaB zmWlaWWZaGrLYi43dW0V3VYy0zK7WNq8sCxHhKIp1GZO7TKNvrw8/H4vz/vlZXCZlYxmVJoM UacR1ApKQkiSN+Su0v6MU645V4ZBMHCGgIoGOwXt9TYE9js5GAw274CusWEEk6/f4lBa0o7g ev9HHO609CFwW40UdHyZC53BUQpaSwooyL3RQMG7oSkMfFeKMbA5d8Mny1cC2i5UY1A6SMHV 0lwsPAYwCFlqabBkLwG/tZyGqf4EaO3zktBU2UqCu3cllFX5KHjsbiWg5b4fg46HFRT02f+Q 0NbygoCxohhov1hIQt1INQVDYxYcLMFRGt43mjFoMUeBwxS25v34TcLzwkYM8m7exqCz5xGC J2c+Y+C0eyloCg5j4HKW4DBR04zAX/SNhlPnQjRczSlCUHDqCgEmXxJMjocvVwYSIOeag4C6 X160dTNvr7Ijvml4FOdNrqP8RPADxbvHzAT/sprjH5R/pHnTk16aNzszeZc1jr/xeBDjr38P kryz9izFO78X03z+t06MH3nzhk5ZuFey8bCoVhlE3erNByVKX7GJTL9LH7PVGKlsZKLyUQTD sYnc7eDADFPsMq67O4RPcyQr51yFX8l8JGFw1jub66rsQfmIYeaxydzT0KppJNgl3Gd/5vS6 lF3HeSuM6J8ylrM5Gmc0EeG8t6BpJpexSVynrYq4gCRmNKsWRao0hjRBpU6K16cqszSqY/GH tGlOFP4fy8mpi/dRoGOHB7EMUsyROtwrlDJSMOiz0jyIY3BFpLTWtVwpkx4Wso6LOu0BXaZa 1HtQDEMo5kt37REPytgjQoaYKorpou5/izER0dkoj1ikXe4jy18dvSQ/0nPPEUjWe9p6Uyrk gQFOPWQYGCnzpHwyVj413jq0Uc5t2yRX9V+6VVOfsSs5dXbq5f2vZUu3ywdjG4joqIlHokxT tFU4fX5f4vjatmYrPdl/YtlueygxVrVgjnnc//PtUu3ahsW+vp2GkzHrvSlbop79FhSEXikk xOE6vfAXpIb+UzsDAAA= X-CFilter-Loop: Reflected Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Now that CONFIG_DEPT_AGGRESSIVE_TIMEOUT_WAIT was introduced, apply the consideration to waitqueue wait, assuming an input 'ret' in ___wait_event() macro is used as a timeout value. Signed-off-by: Byungchul Park --- include/linux/wait.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/include/linux/wait.h b/include/linux/wait.h index aae161e500dc..e77344a8160d 100644 --- a/include/linux/wait.h +++ b/include/linux/wait.h @@ -304,7 +304,7 @@ extern void init_wait_entry(struct wait_queue_entry *wq= _entry, int flags); struct wait_queue_entry __wq_entry; \ long __ret =3D ret; /* explicit shadow */ \ \ - sdt_might_sleep_start(NULL); \ + sdt_might_sleep_start_timeout(NULL, __ret); \ init_wait_entry(&__wq_entry, exclusive ? WQ_FLAG_EXCLUSIVE : 0); \ for (;;) { \ long __int =3D prepare_to_wait_event(&wq_head, &__wq_entry, state);\ --=20 2.17.1 From nobody Fri Dec 19 15:51:27 2025 Received: from invmail4.hynix.com (exvmail4.hynix.com [166.125.252.92]) by smtp.subspace.kernel.org (Postfix) with ESMTP id BE2EA275877; Mon, 19 May 2025 09:18:55 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=166.125.252.92 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747646340; cv=none; b=TMThDHYpAAaIwB7c7W1O8kQVzgjYekHuvVtX5rgLrYKrWZ5ncapg/lYdhPJw4CqiJ7MMuoo9zc3zHAVEHQtd/pZRVGC6t4Lrb4m126ER6bpUkCuEQ+GF1ncItUHNyEXpTr4DdtFBM5zsPTeFNvZrvWmsggx2+GxZGEeA3x/M+zQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747646340; c=relaxed/simple; bh=sXq6ZG2FCPgGrD/93i05yDVFGWV/kuOXrwRc2Zt22go=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References; b=pFaogj0GYg2v/WHeMc10k6Gi+TdbpmqgX+YDPPWRGMVAO7XR/bTzU3AgnEHXuIHamnFlgDDtIcudYjSJYBzBNLnslfVT5a4GN8/+7ytecKHNWzLq8+JGKilp/EfAILbxHm1CPb2HSNousXIE4wXGQ+n8dvHKr5EtxU5SQsz2f5E= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com; spf=pass smtp.mailfrom=sk.com; arc=none smtp.client-ip=166.125.252.92 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=sk.com X-AuditID: a67dfc5b-681ff7000002311f-38-682af76f1e13 From: Byungchul Park To: linux-kernel@vger.kernel.org Cc: kernel_team@skhynix.com, torvalds@linux-foundation.org, damien.lemoal@opensource.wdc.com, linux-ide@vger.kernel.org, adilger.kernel@dilger.ca, linux-ext4@vger.kernel.org, mingo@redhat.com, peterz@infradead.org, will@kernel.org, tglx@linutronix.de, rostedt@goodmis.org, joel@joelfernandes.org, sashal@kernel.org, daniel.vetter@ffwll.ch, duyuyang@gmail.com, johannes.berg@intel.com, tj@kernel.org, tytso@mit.edu, willy@infradead.org, david@fromorbit.com, amir73il@gmail.com, gregkh@linuxfoundation.org, kernel-team@lge.com, linux-mm@kvack.org, akpm@linux-foundation.org, mhocko@kernel.org, minchan@kernel.org, hannes@cmpxchg.org, vdavydov.dev@gmail.com, sj@kernel.org, jglisse@redhat.com, dennis@kernel.org, cl@linux.com, penberg@kernel.org, rientjes@google.com, vbabka@suse.cz, ngupta@vflare.org, linux-block@vger.kernel.org, josef@toxicpanda.com, linux-fsdevel@vger.kernel.org, jack@suse.cz, jlayton@kernel.org, dan.j.williams@intel.com, hch@infradead.org, djwong@kernel.org, dri-devel@lists.freedesktop.org, rodrigosiqueiramelo@gmail.com, melissa.srw@gmail.com, hamohammed.sa@gmail.com, harry.yoo@oracle.com, chris.p.wilson@intel.com, gwan-gyeong.mun@intel.com, max.byungchul.park@gmail.com, boqun.feng@gmail.com, longman@redhat.com, yskelg@gmail.com, yunseong.kim@ericsson.com, yeoreum.yun@arm.com, netdev@vger.kernel.org, matthew.brost@intel.com, her0gyugyu@gmail.com Subject: [PATCH v16 22/42] dept: apply timeout consideration to hashed-waitqueue wait Date: Mon, 19 May 2025 18:18:06 +0900 Message-Id: <20250519091826.19752-23-byungchul@sk.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20250519091826.19752-1-byungchul@sk.com> References: <20250519091826.19752-1-byungchul@sk.com> X-Brightmail-Tracker: H4sIAAAAAAAAAzWSeUiTcRjH+723q8XLut4OMkdW2mkXT5DR8UcvRNFB0QHVaC9ttKlsHikE ztRKU9JQO8ymxhq6bG12N1uKKxfVLLNlamlpSbORueXVsRn98/Dh++X5PP88DC5xkNMYZUy8 oImRqaSUiBD1jitdGPszUrEkLXMm+PpPElB83USBq6oSgalah0FP/UZ44/cgGH72AoeiAheC 0o42HKod7QhsxjQKXn0aD00+LwUNBdkUHC+/TkHj1xEMWgvzMai0bIb3hm4Cnp4pw6Coh4KL RcexwPiCwaChggZDajh0Gi/QMNIRBQ3tzSTYWubD+ZJWCh7YGghw3OnE4NW9YgraTX9IeOp4 QoA/dzq48nJIuPatjIKvfgMOBp+Xhpd2PQYO/WQwpweEmT9+k/A4x45B5pUbGDS9vY+g5uQH DCymZgrqfB4MrJYCHIau1iPozO2lIeP0IA0XdbkIsjMKCUhvXQHDA4HLl/qjQHfZTMC1X81o bTRvKjEhvs7jxfl0axI/5HtN8Ta/nuCdZRx/90IbzafXtNC83pLAW42RfPmDHowv7fORvKXi FMVb+vJpPqu3CeO/PX9Ob52xV7RaLqiUiYJm8ZqDIsXP7M1xufTRLnMemYo6yCzEMBy7nHN3 h2ahkFHs+ZhGBpli53Ju9yAe5InsLM6a0x3IRQzONo/l3lx6i4LFBHYnV+McooNMsOFcpt01 uixmV3J2XRr6Jw3lKs32UVFIIG/JrhvNJewKrqmyhAhKOfZsCFd+q4H8tzCVe2R0E2eQWI/G VCCJMiZRLVOqli9SJMcojy46FKu2oMB3GY6N7LuD+lw7ahHLIOk4sdkWoZCQskRtsroWcQwu nSiusM5TSMRyWXKKoIk9oElQCdpaNJ0hpFPES/1Jcgl7WBYvHBGEOEHzv8WYkGmpaFVHmGqD b3/GhvVzCj3RSWuEz4k6rUPhfye0RVyOniWPWkCeit/2blLrpzr1j0PdtvOO4VBntedc/pF1 J6qI3b0DE+75vnjFYyXftyo33dTUb9mzy9U2+7AzIWe70b0sLy5ijyYs/PMNb2PjAKvWeqLd D1NW9TtT9m7Bwm7LZ3Z5pYRWIYuKxDVa2V8RiASLWQMAAA== X-Brightmail-Tracker: H4sIAAAAAAAAAzWSe0hTcRTH+93nXE0uS/KSRDWTTKkUMg4U1T/VpUjCCkECXXlpQ52xpWZQ +JiWzzJakqYttWVu6twkezgdmiuNzNLUbFlaWcupVG6l2WMa/XP48v1yPge+HAEuvkUuF8gV J3ilQhovoYSEMHxL5vrE70GyEM0MDa7pcwRcrTdQ0FOnR2BoTMfA0bEbBtxOBD+fPMWhWNOD 4PrIaxwabcMILNUZFPS+94Y+1xQFnZo8CjIr6yl4Nj6Hgf3yRQz0pn3wRjdGwOMLFRgUOygo Lc7EPOMTBjO6Ghp0aQEwWl1Cw9xIKHQO95PQXtZJgmUoGK6U2ylotnQSYLszikHvvasUDBv+ kPDY9ogAd6Ef9BQVkFA7WUHBuFuHg841RcNzqxYDm3YZGNUeava33yQ8LLBikF3VgEHfy/sI Ws69xcBk6Keg3eXEwGzS4DB7swPBaOEEDVn5niZK0wsR5GVdJkBtD4OfPzyXy6ZDIf2akYDa X/1oxzbOUG5AXLtzCufU5hRu1vWC4ixuLcF1VbDc3ZLXNKduGaI5rSmJM1cHcZXNDoy7/tVF cqaaHIozfb1Ic7kTfRg32d1N718RJdway8fLk3nlxm0xQtn3vH3HC+mTH4xFZBoaIXORl4Bl NrGOdxkLmmLWsoODM/i89mFWseaCMY8vFOBM/2J2oOwlmg+WMofYlq5Zel4TTACbbe1ZWBYx m1lregb6B13J6o3WBZCXxx/Ka1/wxUwY26cvJy4goRYtqkE+ckVyglQeH7ZBFSdLVchPbjia mGBCngfSnZ4ruoOme3e3IUaAJEtERss6mZiUJqtSE9oQK8AlPqIac6BMLIqVpp7ilYnRyqR4 XtWG/ASExFe0J5KPETPHpCf4OJ4/ziv/p5jAa3kaOr/3WlVb8MoHB+zsVqLjk2FjV+TBX9XS I40o/P3YQHhtROvE9tr84JzASJlll2P8I1MxmqoZjKjXR615F2O2x/rUNaREbzam9H5Z8e3s 7dY1+UOv1DtNluKlse5E3t+5P+Szb0qw0ftS0w1FyZR/eJKv7XDaGX3psHOOXu1ta26SECqZ NDQIV6qkfwEJSOKYPAMAAA== X-CFilter-Loop: Reflected Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Now that CONFIG_DEPT_AGGRESSIVE_TIMEOUT_WAIT was introduced, apply the consideration to hashed-waitqueue wait, assuming an input 'ret' in ___wait_var_event() macro is used as a timeout value. Signed-off-by: Byungchul Park --- include/linux/wait_bit.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/include/linux/wait_bit.h b/include/linux/wait_bit.h index 179a616ad245..9885ac4e1ded 100644 --- a/include/linux/wait_bit.h +++ b/include/linux/wait_bit.h @@ -258,7 +258,7 @@ extern wait_queue_head_t *__var_waitqueue(void *p); struct wait_bit_queue_entry __wbq_entry; \ long __ret =3D ret; /* explicit shadow */ \ \ - sdt_might_sleep_start(NULL); \ + sdt_might_sleep_start_timeout(NULL, __ret); \ init_wait_var_entry(&__wbq_entry, var, \ exclusive ? WQ_FLAG_EXCLUSIVE : 0); \ for (;;) { \ --=20 2.17.1 From nobody Fri Dec 19 15:51:27 2025 Received: from invmail4.hynix.com (exvmail4.hynix.com [166.125.252.92]) by smtp.subspace.kernel.org (Postfix) with ESMTP id E201C27604E; Mon, 19 May 2025 09:18:55 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=166.125.252.92 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747646339; cv=none; b=QPnN09goZqhNII0HTDFuGOFAbTCDNQX5PxYbJsowM5gYLbBEysRiYZMQfhjdFJS8gCo/gJkZNtAIMC6X789cGQKcP3MQkf6w2gSahba3pL7zrGA7CE9b7mbWZiNF7gmJjoD3yC1jQI1WQT6z1TT6CpR8Rnlg3OJ17UuHKsdV0tg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747646339; c=relaxed/simple; bh=WB3yty1UxgRrT5j6RAtqcF8Rog/sCePCJmMWqhpGSYE=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References; b=pYTbSB1E+0CmmxmMC1Yb1RNRP22jJZ53dr2vO3nWLD3XQG4tB8ntw2w4fG7nxg0NBe21TFDeI8x6kiO3QeWWyM/dvZm/4EBBQ90LqDK0VUNQD4Q6gSg0D+vn6YVuSIlRJ6hwd6RCBGOcjH3q3+K7EzBVkwU40A5qeljQf3e8Q3g= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com; spf=pass smtp.mailfrom=sk.com; arc=none smtp.client-ip=166.125.252.92 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=sk.com X-AuditID: a67dfc5b-681ff7000002311f-40-682af76ff51e From: Byungchul Park To: linux-kernel@vger.kernel.org Cc: kernel_team@skhynix.com, torvalds@linux-foundation.org, damien.lemoal@opensource.wdc.com, linux-ide@vger.kernel.org, adilger.kernel@dilger.ca, linux-ext4@vger.kernel.org, mingo@redhat.com, peterz@infradead.org, will@kernel.org, tglx@linutronix.de, rostedt@goodmis.org, joel@joelfernandes.org, sashal@kernel.org, daniel.vetter@ffwll.ch, duyuyang@gmail.com, johannes.berg@intel.com, tj@kernel.org, tytso@mit.edu, willy@infradead.org, david@fromorbit.com, amir73il@gmail.com, gregkh@linuxfoundation.org, kernel-team@lge.com, linux-mm@kvack.org, akpm@linux-foundation.org, mhocko@kernel.org, minchan@kernel.org, hannes@cmpxchg.org, vdavydov.dev@gmail.com, sj@kernel.org, jglisse@redhat.com, dennis@kernel.org, cl@linux.com, penberg@kernel.org, rientjes@google.com, vbabka@suse.cz, ngupta@vflare.org, linux-block@vger.kernel.org, josef@toxicpanda.com, linux-fsdevel@vger.kernel.org, jack@suse.cz, jlayton@kernel.org, dan.j.williams@intel.com, hch@infradead.org, djwong@kernel.org, dri-devel@lists.freedesktop.org, rodrigosiqueiramelo@gmail.com, melissa.srw@gmail.com, hamohammed.sa@gmail.com, harry.yoo@oracle.com, chris.p.wilson@intel.com, gwan-gyeong.mun@intel.com, max.byungchul.park@gmail.com, boqun.feng@gmail.com, longman@redhat.com, yskelg@gmail.com, yunseong.kim@ericsson.com, yeoreum.yun@arm.com, netdev@vger.kernel.org, matthew.brost@intel.com, her0gyugyu@gmail.com Subject: [PATCH v16 23/42] dept: apply timeout consideration to dma fence wait Date: Mon, 19 May 2025 18:18:07 +0900 Message-Id: <20250519091826.19752-24-byungchul@sk.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20250519091826.19752-1-byungchul@sk.com> References: <20250519091826.19752-1-byungchul@sk.com> X-Brightmail-Tracker: H4sIAAAAAAAAAzWSfUzMcRzH+/6e73T8nKcfebzNkJEM+zDMHx6+G/K0MbFx0487ri53pJit SLiUc1u1irqKk7uj3PnDQ9dSpDRE6WkJZUhPlDsilyvzz2evfd57vz7/fDhSXkFP4tQRR0Rd hFKjYKSUtMs/Z572R6BqgdE2A9zfz1FwucDOQPUtGwL7nTgC2h+vg3pPJ4Lfz16QkJZSjSDn /RsS7pS3IHDln2Kg5sNIqHX3MFCZksjA6bwCBl52DBDQnGoiwObYCG8tHymoMuYSkNbOQGba acI3PhPQb7GyYImdCa35GSwMvA+GypY6GlxNcyE9q5mBIlclBeV3WwmouX+ZgRb7IA1V5RUU eJIDoPpSEg03u3MZ6PBYSLC4e1h4VWImoNw8HgrjfcKEPi8NT5JKCEi4epuA2sYHCIrPvSPA Ya9joMzdSYDTkULCr+uPEbQmd7Fw5kI/C5lxyQgSz6RSEN+8GH7/9F2+8j0Y4rILKbj5pw6t WoHtWXaEyzp7SBzvPIZ/uV8z2OUxU/hproDvZbxhcXxxE4vNjqPYmR+I84raCZzT66axw3qe wY5eE4sNXbUE7n7+nN08OVS6PEzUqKNEXdDKvVJVW9s3KrKGi77Wb6JikY01IAkn8IuExrcZ 1H++132dHGKGnyU0NPQP81h+uuBM+kgbkJQj+boRQv2VRjQUjOFDBG9KAjHEFD9T+GAdHGYZ v0TIbikm/0mnCbbCkmGW+PZNiWXDXTm/WKi1ZVFDUoHPkwjGolbmX2Gi8DC/gTIimRn5WZFc HREVrlRrFs1XxUSoo+fv04Y7kO+/LCcHdt1FvdXbShHPIYW/rNA1RyWnlVH6mPBSJHCkYqzM 6pytksvClDHHRZ12j+6oRtSXogCOUkyQLfQcC5PzB5RHxEOiGCnq/qcEJ5kUi/xuNwUpI5d2 XMOH32mmeS+ufnFjWVBu33ZHYExq274QkzajIdrQHJp5VWENGzm6zzBiS5o3YOoY737/9foT j17P+DolcsOO3evPti+LvRW6zvgs8aB2k7Yi1H/DwNo1m1buLB5HP8yr+nSgt+vn1oRRW+tP DlZI9piC5hZM9fvSt3eyPV1B6VXK4EBSp1f+BRl9DwZbAwAA X-Brightmail-Tracker: H4sIAAAAAAAAAzWSe0yTZxjF977flWqXL7XRTxhDmxg3CCrJWB6zxZgsjncYLzNeEvaHNOOb 7SyFtAKyxQTkotLRQE0hKGDFWbGtE1qyeCthVHCIFwZdwQZQwBBRLhmjKIpoYdk/J7+ck+c8 /xyeUjiYSF6rPyIZ9GqdipXRsp1fFMRnvIzVbGrql0No5iQN1VddLHT95kTgasrHMNaWBL2z 4wje3H9IQaW1C8H5oQEKmtoHEXjrj7PQ8/RD8IemWOiwmlgouHCVhb9ezGPor7BgcLp3wGP7 KA2dZXUYKsdYOFtZgMPyDMOc3cGBPW8dDNef4WB+KAE6BgMM+Go6GPAG46Cqtp+FW94OGtqv DWPouVHNwqDrHQOd7X/SMGuOgq7yUgauTNax8GLWToE9NMVBd4sNQ7ttJTQUhluL/11g4E5p C4biXxsx+B/dRNB88gkGtyvAgi80jsHjtlLw+lIbgmHzBAdFv8xxcDbfjMBUVEFDYX8ivHkV /lwzkwD55xpouPI2gLZuIa5aFyK+8SmKFHpyyOvQ3yzxztpocrdOJNfPDHCksDnIEZs7i3jq Y8mFW2OYnJ8OMcTtOMUS97SFIyUTfkwmHzzgdkenyL5Mk3TabMmwcUuqTDMy8g+d2cMfvThn ofOQkytBEbwofCZen7xELTIrrBf7+uaWWCmsET2lo0wJkvGUEFgm9tY8QovBCmGnuGAtxotM C+vEp453SywXPhfPDTZT/5XGiM6GliWOCPtBk2/pViEkin5nLV2GZDb0gQMptfrsdLVWl7jB eFiTq9ce3fB9RrobhRdkPzZffg3N9CS1IoFHquXyBu+nGgWjzjbmprcikadUSrnD84lGIU9T 5/4kGTIOGrJ0krEVRfG0apU8+YCUqhAOqY9IhyUpUzL8n2I+IjIP3Vyetno93nPAOjQTQy37 rnhHzr62r76pHtobF/ew21Fpet4bv9b76rH0bH+X/sfT9HBZ49fmy6sWMjb+sDm13uI/mHL7 xPTLPxp/V54o6I5auysnhSTf69tT9FHWxKbgDZmvc5t5u6nq458339EFy/aNDuRHKy3P0yMD bcFjmsxvV8TnqmijRp0QSxmM6vdu8+cZPQMAAA== X-CFilter-Loop: Reflected Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Now that CONFIG_DEPT_AGGRESSIVE_TIMEOUT_WAIT was introduced, apply the consideration to dma fence wait. Signed-off-by: Byungchul Park --- drivers/dma-buf/dma-fence.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/drivers/dma-buf/dma-fence.c b/drivers/dma-buf/dma-fence.c index 5d2fd2f6a46d..a45e5416f2dd 100644 --- a/drivers/dma-buf/dma-fence.c +++ b/drivers/dma-buf/dma-fence.c @@ -784,7 +784,7 @@ dma_fence_default_wait(struct dma_fence *fence, bool in= tr, signed long timeout) cb.task =3D current; list_add(&cb.base.node, &fence->cb_list); =20 - sdt_might_sleep_start(NULL); + sdt_might_sleep_start_timeout(NULL, timeout); while (!test_bit(DMA_FENCE_FLAG_SIGNALED_BIT, &fence->flags) && ret > 0) { if (intr) __set_current_state(TASK_INTERRUPTIBLE); @@ -888,7 +888,7 @@ dma_fence_wait_any_timeout(struct dma_fence **fences, u= int32_t count, } } =20 - sdt_might_sleep_start(NULL); + sdt_might_sleep_start_timeout(NULL, timeout); while (ret > 0) { if (intr) set_current_state(TASK_INTERRUPTIBLE); --=20 2.17.1 From nobody Fri Dec 19 15:51:27 2025 Received: from invmail4.hynix.com (exvmail4.hynix.com [166.125.252.92]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 5E9752777F5; Mon, 19 May 2025 09:18:56 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=166.125.252.92 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747646340; cv=none; b=qLEZofsRZnRjJ6DmZ7EPVbrxMRpyaA+eRdt2B8YM7oEo33tvTmVnz+khCsRFTzv+av1hCWkaKz8ES+Et+YxOZpWpY+VBO4oLw3ZCawAuLkpqRI9BVz4GUr0Dbt+xc8w6iK/cNv+1xXz0efMY9ZIS3a1skHmyc6L7GRM0wlJZpSg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747646340; c=relaxed/simple; bh=zw9buFd2EHMbKRLJjS8zYwIdV1jL6Jp00aCB8bfg1j0=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References; b=JfqCPeZ0lMxq69DSOKBardXUbHq3ojP7DBqbAYXIq/OUsjC09i8qKq8KFz/ljIetzRQqQRiAIqnfy9xC1oWtAMjZ8fQWM8f+mLhpMeZhYUjOLZULtW0VwG7Zi8Wf5KFXkWbERZq0k7yYD6j8JOCixHKojQpVv04OFpYSGKTch3Q= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com; spf=pass smtp.mailfrom=sk.com; arc=none smtp.client-ip=166.125.252.92 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=sk.com X-AuditID: a67dfc5b-681ff7000002311f-4f-682af76fff2b From: Byungchul Park To: linux-kernel@vger.kernel.org Cc: kernel_team@skhynix.com, torvalds@linux-foundation.org, damien.lemoal@opensource.wdc.com, linux-ide@vger.kernel.org, adilger.kernel@dilger.ca, linux-ext4@vger.kernel.org, mingo@redhat.com, peterz@infradead.org, will@kernel.org, tglx@linutronix.de, rostedt@goodmis.org, joel@joelfernandes.org, sashal@kernel.org, daniel.vetter@ffwll.ch, duyuyang@gmail.com, johannes.berg@intel.com, tj@kernel.org, tytso@mit.edu, willy@infradead.org, david@fromorbit.com, amir73il@gmail.com, gregkh@linuxfoundation.org, kernel-team@lge.com, linux-mm@kvack.org, akpm@linux-foundation.org, mhocko@kernel.org, minchan@kernel.org, hannes@cmpxchg.org, vdavydov.dev@gmail.com, sj@kernel.org, jglisse@redhat.com, dennis@kernel.org, cl@linux.com, penberg@kernel.org, rientjes@google.com, vbabka@suse.cz, ngupta@vflare.org, linux-block@vger.kernel.org, josef@toxicpanda.com, linux-fsdevel@vger.kernel.org, jack@suse.cz, jlayton@kernel.org, dan.j.williams@intel.com, hch@infradead.org, djwong@kernel.org, dri-devel@lists.freedesktop.org, rodrigosiqueiramelo@gmail.com, melissa.srw@gmail.com, hamohammed.sa@gmail.com, harry.yoo@oracle.com, chris.p.wilson@intel.com, gwan-gyeong.mun@intel.com, max.byungchul.park@gmail.com, boqun.feng@gmail.com, longman@redhat.com, yskelg@gmail.com, yunseong.kim@ericsson.com, yeoreum.yun@arm.com, netdev@vger.kernel.org, matthew.brost@intel.com, her0gyugyu@gmail.com Subject: [PATCH v16 24/42] dept: make dept able to work with an external wgen Date: Mon, 19 May 2025 18:18:08 +0900 Message-Id: <20250519091826.19752-25-byungchul@sk.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20250519091826.19752-1-byungchul@sk.com> References: <20250519091826.19752-1-byungchul@sk.com> X-Brightmail-Tracker: H4sIAAAAAAAAAzWSXUwTeRTF/c/8OzOt1EyqWUYkWdOIH+i6y4rujaLBFx2N+Ak+qFGrTLaN ULAVBBMVtBAFIUpSiKhYKFZCq9TWBwSLLCzdBSOya6UFAYW4KAEkQVrBgtpi9uXm5Jx7fvfl MqTMKQpjVOpTgkatSJJTEiwZDSn/KeVTpPIXnXkVeCcuYbhZY6Gg474ZgeVhNgFDLVvB7RtB 4H/2nIQSfQeC8v5eEh46+xA4qi5Q8OLtPHB5xyho1edTcNFYQ8E/w9ME9BQXEWC2xcFr0yCG p1crCCgZouBGyUUiMN4TMGWqpsGUFQEDVaU0TPdHQWtfpwgc3SvhelkPBY8drRictQMEvKi7 SUGf5asInjr/xuArXAQd1wpEcO9DBQXDPhMJJu8YDf82GghwGn4Aqy4AzP34RQR/FTQSkFv5 gABXVz2ChktvCLBZOilo9o4QYLfpSfh8twXBQOEoDTlXpmi4kV2IID+nGIOuZy34JwOXb01E QfZtK4Z7M50odiNvKbMgvnlkjOR19tP8Z+9Linf4DJhvq+D4R6W9NK9r6KZ5gy2Nt1dF8sbH QwRfPu4V8bbqyxRvGy+i+bxRF8F/aG+nd4cfkMQkCkmqdEHz86ajEmWRtx6nNsVmOJ6/wlno UXQeEjMcG829G3SL8hAzqyfacdCm2GWcxzNFBvUCdjFnLxgMrEgYku2cy7lvdaFgMJ+N44yv RumgxmwE9+eEmw5ypOw6zvhy33f8j5zZ2jjLEQfs7vzm2aqMXcu5zGU4yOTY22Ju2viJ/F5Y yP1R5cFXkdSA5lQjmUqdnqxQJUWvVmaqVRmrj6ck21Dgu0xnpw/WovGOfU2IZZA8RGp1rFDK RIp0bWZyE+IYUr5AWm1frpRJExWZZwRNyhFNWpKgbUKLGCwPlf7qO50oY39XnBJOCEKqoPk/ JRhxWBYyrSjZiQ+viZtsi97Qv2R7TM1M7OaluUfePqEaEvQ9fr06KiLcE3sntBL8Mqbl0JK9 tfvl6wvTtJc3x8S7DmzxCDXxYwkni55khe8Jy3MM/zbT9i6n+FyO7r+p8+e7GRR6Ylt47a66 UvsO3ySIn1X4e0MMHyvXWa5lWN1rNlmOqeRYq1RERZIareIbCm5zg1kDAAA= X-Brightmail-Tracker: H4sIAAAAAAAAAzWSb0xTZxjF9773L52dN5XoFXQuTYjKAo4E9DEaw/zCu00XFz84zYx048Y2 /E2raE2MINVoEaIsyAApBZbK2jpraxSUS7BFZjFDFCxSAZUsZEQqibRVlKmtyb6cnJyT53e+ PDylameSeF3RAUlfpClQswpa8f2mirTiV6nar1r9SyESPkXDhctOFgb/dCBwXi3HMH07B0ai Mwje/n2PgrraQQQtz8YpuNo3gUBuP87C0D+fwXBklgV/bSULFW2XWbj/fAHD2PkaDA73dnhi m6Lh7tlWDHXTLDTWVeCY/Ith3mbnwFaWApPtDRwsPMsA/0SAAV+TnwE5+CXUW8ZY6JL9NPR1 TGIYunGBhQnnewbu9t2hIVqdDIPnqhi49KKVhedRGwW2yCwHD3qsGPqsS8FlilFPzr1j4K+q Hgwnf7+CYXj0JoLuU08xuJ0BFnyRGQwedy0Fby7eRjBZHeLgxJl5DhrLqxFUnjhPg2ksC96+ ji03hTOgvNlFw6X/Aih7C3FanIj4ZmYpYvIcIm8iD1kiR6006W8VSWfDOEdM3UGOWN0Hiac9 lbR1TWPS8jLCELf9NEvcL2s4Yg4NY/JiYIDbsXKPYnOeVKArlfTrtuQqtDWRm3SJN/uwfO8x XYY6M82I50UhUwwP0GaUwLPCavHRo3kq7hOFL0RP1RRjRgqeEgKfiiNNoyheLBG2i22PQ1zc 00KK2Bse4eIcpbBebHu4Mx6LwirR4er5yEmIxcFK38dTlZAlDjss9FmksKJP7ChRV1RaqNEV ZKUb8rXGIt3h9F+KC90o9j+2owvnOlB4KMeLBB6pFyld8lqtitGUGoyFXiTylDpRafes0aqU eRrjEUlfvE9/sEAyeFEyT6uXKb/dJeWqhP2aA1K+JJVI+v9bzCcklaHKueVrtmoHS/gzu24l TTUv+2b3hj1dwd/WTfN/3P+J3PKFNA3jUtXnuVZyozmlY23mlY32TvOrvf3bwpZjI+LPSXJg 66glPWw25igTX/8gK6/j7t6NvUFm7sh+U8biih+f9LvSfs1f0ln/tSNkzWvxfvd+8Z13ZddW yCvSqCg2PlDTBq0mI5XSGzQfAAmSBF07AwAA X-CFilter-Loop: Reflected Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" There is a case where the total map size of waits of a class is so large. For instance, PG_locked is the case if every struct page embeds its regular map for PG_locked. The total size for the maps will be 'the # of pages * sizeof(struct dept_map)', which is too big to accept. Keep the minimum data in the case, timestamp called 'wgen', that dept uses. Make dept able to work with the wgen instead of whole regular map. Signed-off-by: Byungchul Park --- include/linux/dept.h | 18 ++++++++++++++---- include/linux/dept_sdt.h | 6 +++--- kernel/dependency/dept.c | 30 +++++++++++++++++++++--------- 3 files changed, 38 insertions(+), 16 deletions(-) diff --git a/include/linux/dept.h b/include/linux/dept.h index 49f457390521..236e4f06e5c8 100644 --- a/include/linux/dept.h +++ b/include/linux/dept.h @@ -372,6 +372,13 @@ struct dept_wait_hist { unsigned int ctxt_id; }; =20 +/* + * for subsystems that requires compact use of memory e.g. struct page + */ +struct dept_ext_wgen { + unsigned int wgen; +}; + extern void dept_on(void); extern void dept_off(void); extern void dept_init(void); @@ -381,6 +388,7 @@ extern void dept_free_range(void *start, unsigned int s= z); =20 extern void dept_map_init(struct dept_map *m, struct dept_key *k, int sub_= u, const char *n); extern void dept_map_reinit(struct dept_map *m, struct dept_key *k, int su= b_u, const char *n); +extern void dept_ext_wgen_init(struct dept_ext_wgen *ewg); extern void dept_map_copy(struct dept_map *to, struct dept_map *from); extern void dept_wait(struct dept_map *m, unsigned long w_f, unsigned long= ip, const char *w_fn, int sub_l, long timeout); extern void dept_stage_wait(struct dept_map *m, struct dept_key *k, unsign= ed long ip, const char *w_fn, long timeout); @@ -389,8 +397,8 @@ extern void dept_clean_stage(void); extern void dept_ttwu_stage_wait(struct task_struct *t, unsigned long ip); extern void dept_ecxt_enter(struct dept_map *m, unsigned long e_f, unsigne= d long ip, const char *c_fn, const char *e_fn, int sub_l); extern bool dept_ecxt_holding(struct dept_map *m, unsigned long e_f); -extern void dept_request_event(struct dept_map *m); -extern void dept_event(struct dept_map *m, unsigned long e_f, unsigned lon= g ip, const char *e_fn); +extern void dept_request_event(struct dept_map *m, struct dept_ext_wgen *e= wg); +extern void dept_event(struct dept_map *m, unsigned long e_f, unsigned lon= g ip, const char *e_fn, struct dept_ext_wgen *ewg); extern void dept_ecxt_exit(struct dept_map *m, unsigned long e_f, unsigned= long ip); extern void dept_sched_enter(void); extern void dept_sched_exit(void); @@ -417,6 +425,7 @@ extern void dept_hardirqs_off(void); #else /* !CONFIG_DEPT */ struct dept_key { }; struct dept_map { }; +struct dept_ext_wgen { }; =20 #define DEPT_MAP_INITIALIZER(n, k) { } =20 @@ -429,6 +438,7 @@ struct dept_map { }; =20 #define dept_map_init(m, k, su, n) do { (void)(n); (void)(k); } while (0) #define dept_map_reinit(m, k, su, n) do { (void)(n); (void)(k); } while = (0) +#define dept_ext_wgen_init(wg) do { } while (0) #define dept_map_copy(t, f) do { } while (0) #define dept_wait(m, w_f, ip, w_fn, sl, t) do { (void)(w_fn); } while (0) #define dept_stage_wait(m, k, ip, w_fn, t) do { (void)(k); (void)(w_fn); = } while (0) @@ -437,8 +447,8 @@ struct dept_map { }; #define dept_ttwu_stage_wait(t, ip) do { } while (0) #define dept_ecxt_enter(m, e_f, ip, c_fn, e_fn, sl) do { (void)(c_fn); (vo= id)(e_fn); } while (0) #define dept_ecxt_holding(m, e_f) false -#define dept_request_event(m) do { } while (0) -#define dept_event(m, e_f, ip, e_fn) do { (void)(e_fn); } while (0) +#define dept_request_event(m, wg) do { } while (0) +#define dept_event(m, e_f, ip, e_fn, wg) do { (void)(e_fn); } while (0) #define dept_ecxt_exit(m, e_f, ip) do { } while (0) #define dept_sched_enter() do { } while (0) #define dept_sched_exit() do { } while (0) diff --git a/include/linux/dept_sdt.h b/include/linux/dept_sdt.h index 14917df0cc30..9cd70affaf35 100644 --- a/include/linux/dept_sdt.h +++ b/include/linux/dept_sdt.h @@ -25,7 +25,7 @@ =20 #define sdt_wait_timeout(m, t) \ do { \ - dept_request_event(m); \ + dept_request_event(m, NULL); \ dept_wait(m, 1UL, _THIS_IP_, __func__, 0, t); \ } while (0) #define sdt_wait(m) sdt_wait_timeout(m, -1L) @@ -49,9 +49,9 @@ #define sdt_might_sleep_end() dept_clean_stage() =20 #define sdt_ecxt_enter(m) dept_ecxt_enter(m, 1UL, _THIS_IP_, "start", "ev= ent", 0) -#define sdt_event(m) dept_event(m, 1UL, _THIS_IP_, __func__) +#define sdt_event(m) dept_event(m, 1UL, _THIS_IP_, __func__, NULL) #define sdt_ecxt_exit(m) dept_ecxt_exit(m, 1UL, _THIS_IP_) -#define sdt_request_event(m) dept_request_event(m) +#define sdt_request_event(m) dept_request_event(m, NULL) #else /* !CONFIG_DEPT */ #define sdt_map_init(m) do { } while (0) #define sdt_map_init_key(m, k) do { (void)(k); } while (0) diff --git a/kernel/dependency/dept.c b/kernel/dependency/dept.c index a6abba8f3a2c..79357a3a03bb 100644 --- a/kernel/dependency/dept.c +++ b/kernel/dependency/dept.c @@ -2172,6 +2172,11 @@ void dept_map_reinit(struct dept_map *m, struct dept= _key *k, int sub_u, } EXPORT_SYMBOL_GPL(dept_map_reinit); =20 +void dept_ext_wgen_init(struct dept_ext_wgen *ewg) +{ + ewg->wgen =3D 0U; +} + void dept_map_copy(struct dept_map *to, struct dept_map *from) { if (unlikely(!dept_working())) { @@ -2355,7 +2360,7 @@ static void __dept_wait(struct dept_map *m, unsigned = long w_f, */ static void __dept_event(struct dept_map *m, struct dept_map *real_m, unsigned long e_f, unsigned long ip, const char *e_fn, - bool sched_map) + bool sched_map, unsigned int wg) { struct dept_class *c; struct dept_key *k; @@ -2377,7 +2382,7 @@ static void __dept_event(struct dept_map *m, struct d= ept_map *real_m, c =3D check_new_class(&m->map_key, k, sub_id(m, e), m->name, sched_map); =20 if (c) - do_event(m, real_m, c, READ_ONCE(m->wgen), ip, e_fn); + do_event(m, real_m, c, wg, ip, e_fn); } =20 void dept_wait(struct dept_map *m, unsigned long w_f, @@ -2602,7 +2607,7 @@ void dept_ttwu_stage_wait(struct task_struct *request= or, unsigned long ip) if (!m.keys) goto exit; =20 - __dept_event(&m, real_m, 1UL, ip, "try_to_wake_up", sched_map); + __dept_event(&m, real_m, 1UL, ip, "try_to_wake_up", sched_map, m.wgen); exit: dept_exit(flags); } @@ -2781,10 +2786,11 @@ bool dept_ecxt_holding(struct dept_map *m, unsigned= long e_f) } EXPORT_SYMBOL_GPL(dept_ecxt_holding); =20 -void dept_request_event(struct dept_map *m) +void dept_request_event(struct dept_map *m, struct dept_ext_wgen *ewg) { unsigned long flags; unsigned int wg; + unsigned int *wg_p; =20 if (unlikely(!dept_working())) return; @@ -2797,18 +2803,22 @@ void dept_request_event(struct dept_map *m) */ flags =3D dept_enter_recursive(); =20 + wg_p =3D ewg ? &ewg->wgen : &m->wgen; + wg =3D next_wgen(); - WRITE_ONCE(m->wgen, wg); + WRITE_ONCE(*wg_p, wg); =20 dept_exit_recursive(flags); } EXPORT_SYMBOL_GPL(dept_request_event); =20 void dept_event(struct dept_map *m, unsigned long e_f, - unsigned long ip, const char *e_fn) + unsigned long ip, const char *e_fn, + struct dept_ext_wgen *ewg) { struct dept_task *dt =3D dept_task(); unsigned long flags; + unsigned int *wg_p; =20 if (unlikely(!dept_working())) return; @@ -2816,24 +2826,26 @@ void dept_event(struct dept_map *m, unsigned long e= _f, if (m->nocheck) return; =20 + wg_p =3D ewg ? &ewg->wgen : &m->wgen; + if (dt->recursive) { /* * Dept won't work with this even though an event * context has been asked. Don't make it confused at * handling the event. Disable it until the next. */ - WRITE_ONCE(m->wgen, 0U); + WRITE_ONCE(*wg_p, 0U); return; } =20 flags =3D dept_enter(); =20 - __dept_event(m, m, e_f, ip, e_fn, false); + __dept_event(m, m, e_f, ip, e_fn, false, READ_ONCE(*wg_p)); =20 /* * Keep the map diabled until the next sleep. */ - WRITE_ONCE(m->wgen, 0U); + WRITE_ONCE(*wg_p, 0U); =20 dept_exit(flags); } --=20 2.17.1 From nobody Fri Dec 19 15:51:27 2025 Received: from invmail4.hynix.com (exvmail4.hynix.com [166.125.252.92]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 093762701C8; Mon, 19 May 2025 09:18:56 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=166.125.252.92 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747646341; cv=none; b=sYFyhOJRMZqGbTqYYj0JgvCbwPUpLeXYbvuz31V3M+R3BuMsEy6WjmYyV+umBHnwqC75tBLKgZwRKhgYhKFf09c0jkQxq8H/vorydn8STKtDmt6QvHErJWI/9stc5q5o6x6a83IEUusStH3Q9Kw43xJiOfsI8tTEg140l77YWqo= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747646341; c=relaxed/simple; bh=eqhIu4Ytfemwyxgng3iHH4sQt4GfdE3Wth8zaCAgCgk=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References; b=f3rnHNbmzAtyTv7fivOFuDTJH8PpRHpn9OZRJsD9WZ3uM4e16OB3BRzZHMiRwCMEhSgUxp1yGeO42ypLHC/bLEKkPNQ9kKxWuz4PWbzCZ4no6yttJvICk29oG80D1xDYHV1h9i+CyRFPmlzYv7/n61uwrTL1qHIdY0zkUUdEdXM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com; spf=pass smtp.mailfrom=sk.com; arc=none smtp.client-ip=166.125.252.92 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=sk.com X-AuditID: a67dfc5b-681ff7000002311f-5e-682af76f6d09 From: Byungchul Park To: linux-kernel@vger.kernel.org Cc: kernel_team@skhynix.com, torvalds@linux-foundation.org, damien.lemoal@opensource.wdc.com, linux-ide@vger.kernel.org, adilger.kernel@dilger.ca, linux-ext4@vger.kernel.org, mingo@redhat.com, peterz@infradead.org, will@kernel.org, tglx@linutronix.de, rostedt@goodmis.org, joel@joelfernandes.org, sashal@kernel.org, daniel.vetter@ffwll.ch, duyuyang@gmail.com, johannes.berg@intel.com, tj@kernel.org, tytso@mit.edu, willy@infradead.org, david@fromorbit.com, amir73il@gmail.com, gregkh@linuxfoundation.org, kernel-team@lge.com, linux-mm@kvack.org, akpm@linux-foundation.org, mhocko@kernel.org, minchan@kernel.org, hannes@cmpxchg.org, vdavydov.dev@gmail.com, sj@kernel.org, jglisse@redhat.com, dennis@kernel.org, cl@linux.com, penberg@kernel.org, rientjes@google.com, vbabka@suse.cz, ngupta@vflare.org, linux-block@vger.kernel.org, josef@toxicpanda.com, linux-fsdevel@vger.kernel.org, jack@suse.cz, jlayton@kernel.org, dan.j.williams@intel.com, hch@infradead.org, djwong@kernel.org, dri-devel@lists.freedesktop.org, rodrigosiqueiramelo@gmail.com, melissa.srw@gmail.com, hamohammed.sa@gmail.com, harry.yoo@oracle.com, chris.p.wilson@intel.com, gwan-gyeong.mun@intel.com, max.byungchul.park@gmail.com, boqun.feng@gmail.com, longman@redhat.com, yskelg@gmail.com, yunseong.kim@ericsson.com, yeoreum.yun@arm.com, netdev@vger.kernel.org, matthew.brost@intel.com, her0gyugyu@gmail.com Subject: [PATCH v16 25/42] dept: track PG_locked with dept Date: Mon, 19 May 2025 18:18:09 +0900 Message-Id: <20250519091826.19752-26-byungchul@sk.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20250519091826.19752-1-byungchul@sk.com> References: <20250519091826.19752-1-byungchul@sk.com> X-Brightmail-Tracker: H4sIAAAAAAAAAzWSa0yTZxTHfd47leprR9wrLrrVuDlUFIJ4YtQtJmaPLrol24dlftAqb2y1 XFIUxUQtcgmDQeqlVLFiC6MyWqC2xKCjBGFUwBsqtsCgm3jZkJuptgiCrmD8cvLL/5/zO18O R8pa6UhOlXRA1CQp1HJGQkmGw80rk8eilKufemMh8CqXAmONjYGOaisCW20GAQMt34A3OITg ze27JBj0HQjMj/pIqHX7ELgqTjDw4Mkc6AyMMtCmz2cgs6yGgXuDkwT0Fp0iwOrYBn9bnlFw U1dKgGGAgfOGTCI0/iNg3FLJgkW7FPorilmYfBQDbT4PDa6e5XCupJeBelcbBe66fgIeXDMy 4LO9o+Gmu5WCYOFC6DhZQEPVSCkDg0ELCZbAKAv3G00EuE3zwZ4VEua8fEvDjYJGAnJ+u0xA Z/cfCBpy/yHAYfMw0BwYIsDp0JMwcakFQX/hMAvZv46zcD6jEEF+dhEFWb1r4M3r0OULr2Ig 46KdgqopD/p6A7aV2BBuHholcZbzEJ4IPGSwK2iicHupgK8W97E4q6GHxSbHQeysiMJl9QME NvsDNHZU/sJgh/8Ui/OGOwk8cucO+/0nP0vWJ4hqVZqoWbVxl0SZO2WnU1p3Hb7fP0Jpkfa7 PBTGCXycMNB+lv3AZTdezDDDfyF0dY2T0xzBfyo4C57ReUjCkbxntuC90I2mi4/4dYJ7TEtM M8UvFVo8ZTO5lI8XBnuL0HvpYsFqb5wRhYXynvzmmVzGrxE6rSXUtFTgDWFC362n1PuFBcL1 ii5Kh6QmNKsSyVRJaYkKlTouWpmepDocvSc50YFC/2U5OrmjDvk7fmhCPIfk4VK760uljFak paYnNiGBI+UR0krnMqVMmqBIPyJqkndqDqrF1Ca0kKPkH0tjg4cSZPxexQFxvyimiJoPLcGF RWqRMv6zl4bN11ui8x47Pk83Z24pNUSW7710zHiuTle+qJosVv+0ZEVaOYvO3HV7rUZdyfzw Iz7SQ/nNJ/XGTc+9l7trt85r3z5RNfeKd/W3urHj3O7YiW27F8dn/5mjkqwt/H3ljziY0kPr j/v/2jp1NqLq39NxmmjN9iW+r07vGNv3Qk6lKhUxUaQmVfE/ra0sz1sDAAA= X-Brightmail-Tracker: H4sIAAAAAAAAAzWSfUzMcRzHfX+P13H2cxq/iWlnqExkZR8P87jlJ48bZvOPTn78btVld0kZ cymkujxsFaVcxcndxXWnrXC0TqdjSOV6WIVjTStldEcP4sr889lrr/f2fv/zEeFSAzlHpFAm 8iqlPE5GiQnxzjVpSxN+hgjL7U1zwTOUQcCN+yYKGu8ZEZgepGLQW78FWr39CEZfvcEhP7cR QcnHLhweOLoR2MrPUtD8eTq0eAYpcOZmUZBWdp+Ct31jGHTmXcXAaNkB7/U9BLy8XIpBfi8F hflpmO98wWBYb6BBr1kI7vICGsY+hoGz20WCvchJgq1jCVwv7qTgsc1JgKPajUHzwxsUdJv+ kPDS0UCANycAGq9oSagYKKWgz6vHQe8ZpKGpVoeBQzcLzOm+1vM/xkl4rq3F4PytSgxa2h8h eJLxAQOLyUWB3dOPgdWSi8PInXoE7pyvNJzLHqahMDUHQda5PALSOyNg9JdvuWgoDFJvmgmo +O1CG9ZxpmIT4uz9gziXbj3BjXjeUZzNqyO4F6UsV1PQRXPpTzpoTmc5zlnLQ7iyx70YV/Ld Q3IWw0WKs3y/SnOZX1swbuD1a3r3vAPitYf5OEUSr1q2LlosZPw2k8caopOb3AOEBml2ZSI/ EcuEs2XPv9ETTDGL2ba2YXyC/ZlA1qrtITORWIQzrqlsa1E7mghmMqtZx08NNsEEs5Ctd5VN egmzku3rzEP/SuezRnPtZJGfz3dk2Se9lIlgW4zFxGUk1qEpBuSvUCbFyxVxEaHqWCFFqUgO jUmItyDfB+lPj12pRkPNW+oQI0KyaRKzLViQkvIkdUp8HWJFuMxfYrAGCVLJYXnKSV6VcFB1 PI5X16EAESGbLYnaz0dLmaPyRD6W54/xqv8pJvKbo0F3nSujSgvnx4bbosJ7cjpqPux1r254 FLlnVd72bxJlyubI9cLGS9JnimBvW8TW7HtFrk8a4Vp1zbXAqhkzTNqqI2zyoikLNgZqRzzj 43Vd/TFrbey7bfSzvWTYvqdLrWOHTjQKlZtyE0tODQeciQw9WWG7XWnUXUgaDcpiVhy8dDRb RqgFeVgIrlLL/wK/9ilkPQMAAA== X-CFilter-Loop: Reflected Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Makes dept able to track PG_locked waits and events, which will be useful in practice. See the following link that shows dept worked with PG_locked and detected real issues in practice: https://lore.kernel.org/lkml/1674268856-31807-1-git-send-email-byungchul= .park@lge.com/ Signed-off-by: Byungchul Park --- include/linux/mm_types.h | 2 + include/linux/page-flags.h | 125 +++++++++++++++++++++++++++++++++---- include/linux/pagemap.h | 16 ++++- mm/filemap.c | 26 ++++++++ mm/mm_init.c | 2 + 5 files changed, 158 insertions(+), 13 deletions(-) diff --git a/include/linux/mm_types.h b/include/linux/mm_types.h index 56d07edd01f9..7e9d63cef28a 100644 --- a/include/linux/mm_types.h +++ b/include/linux/mm_types.h @@ -20,6 +20,7 @@ #include #include #include +#include =20 #include =20 @@ -224,6 +225,7 @@ struct page { struct page *kmsan_shadow; struct page *kmsan_origin; #endif + struct dept_ext_wgen pg_locked_wgen; } _struct_page_alignment; =20 /* diff --git a/include/linux/page-flags.h b/include/linux/page-flags.h index e6a21b62dcce..73cb8a1ad4f3 100644 --- a/include/linux/page-flags.h +++ b/include/linux/page-flags.h @@ -194,6 +194,61 @@ enum pageflags { =20 #ifndef __GENERATING_BOUNDS_H =20 +#ifdef CONFIG_DEPT +#include +#include + +extern struct dept_map pg_locked_map; + +/* + * Place the following annotations in its suitable point in code: + * + * Annotate dept_page_set_bit() around firstly set_bit*() + * Annotate dept_page_clear_bit() around clear_bit*() + * Annotate dept_page_wait_on_bit() around wait_on_bit*() + */ + +static inline void dept_page_set_bit(struct page *p, int bit_nr) +{ + if (bit_nr =3D=3D PG_locked) + dept_request_event(&pg_locked_map, &p->pg_locked_wgen); +} + +static inline void dept_page_clear_bit(struct page *p, int bit_nr) +{ + if (bit_nr =3D=3D PG_locked) + dept_event(&pg_locked_map, 1UL, _RET_IP_, __func__, &p->pg_locked_wgen); +} + +static inline void dept_page_wait_on_bit(struct page *p, int bit_nr) +{ + if (bit_nr =3D=3D PG_locked) + dept_wait(&pg_locked_map, 1UL, _RET_IP_, __func__, 0, -1L); +} + +static inline void dept_folio_set_bit(struct folio *f, int bit_nr) +{ + dept_page_set_bit(&f->page, bit_nr); +} + +static inline void dept_folio_clear_bit(struct folio *f, int bit_nr) +{ + dept_page_clear_bit(&f->page, bit_nr); +} + +static inline void dept_folio_wait_on_bit(struct folio *f, int bit_nr) +{ + dept_page_wait_on_bit(&f->page, bit_nr); +} +#else +#define dept_page_set_bit(p, bit_nr) do { } while (0) +#define dept_page_clear_bit(p, bit_nr) do { } while (0) +#define dept_page_wait_on_bit(p, bit_nr) do { } while (0) +#define dept_folio_set_bit(f, bit_nr) do { } while (0) +#define dept_folio_clear_bit(f, bit_nr) do { } while (0) +#define dept_folio_wait_on_bit(f, bit_nr) do { } while (0) +#endif + #ifdef CONFIG_HUGETLB_PAGE_OPTIMIZE_VMEMMAP DECLARE_STATIC_KEY_FALSE(hugetlb_optimize_vmemmap_key); =20 @@ -415,27 +470,51 @@ static __always_inline bool folio_test_##name(const s= truct folio *folio) \ =20 #define FOLIO_SET_FLAG(name, page) \ static __always_inline void folio_set_##name(struct folio *folio) \ -{ set_bit(PG_##name, folio_flags(folio, page)); } +{ \ + set_bit(PG_##name, folio_flags(folio, page)); \ + dept_folio_set_bit(folio, PG_##name); \ +} =20 #define FOLIO_CLEAR_FLAG(name, page) \ static __always_inline void folio_clear_##name(struct folio *folio) \ -{ clear_bit(PG_##name, folio_flags(folio, page)); } +{ \ + clear_bit(PG_##name, folio_flags(folio, page)); \ + dept_folio_clear_bit(folio, PG_##name); \ +} =20 #define __FOLIO_SET_FLAG(name, page) \ static __always_inline void __folio_set_##name(struct folio *folio) \ -{ __set_bit(PG_##name, folio_flags(folio, page)); } +{ \ + __set_bit(PG_##name, folio_flags(folio, page)); \ + dept_folio_set_bit(folio, PG_##name); \ +} =20 #define __FOLIO_CLEAR_FLAG(name, page) \ static __always_inline void __folio_clear_##name(struct folio *folio) \ -{ __clear_bit(PG_##name, folio_flags(folio, page)); } +{ \ + __clear_bit(PG_##name, folio_flags(folio, page)); \ + dept_folio_clear_bit(folio, PG_##name); \ +} =20 #define FOLIO_TEST_SET_FLAG(name, page) \ static __always_inline bool folio_test_set_##name(struct folio *folio) \ -{ return test_and_set_bit(PG_##name, folio_flags(folio, page)); } +{ \ + bool __ret =3D test_and_set_bit(PG_##name, folio_flags(folio, page)); \ + \ + if (!__ret) \ + dept_folio_set_bit(folio, PG_##name); \ + return __ret; \ +} =20 #define FOLIO_TEST_CLEAR_FLAG(name, page) \ static __always_inline bool folio_test_clear_##name(struct folio *folio) \ -{ return test_and_clear_bit(PG_##name, folio_flags(folio, page)); } +{ \ + bool __ret =3D test_and_clear_bit(PG_##name, folio_flags(folio, page)); \ + \ + if (__ret) \ + dept_folio_clear_bit(folio, PG_##name); \ + return __ret; \ +} =20 #define FOLIO_FLAG(name, page) \ FOLIO_TEST_FLAG(name, page) \ @@ -450,32 +529,54 @@ static __always_inline int Page##uname(const struct p= age *page) \ #define SETPAGEFLAG(uname, lname, policy) \ FOLIO_SET_FLAG(lname, FOLIO_##policy) \ static __always_inline void SetPage##uname(struct page *page) \ -{ set_bit(PG_##lname, &policy(page, 1)->flags); } +{ \ + set_bit(PG_##lname, &policy(page, 1)->flags); \ + dept_page_set_bit(page, PG_##lname); \ +} =20 #define CLEARPAGEFLAG(uname, lname, policy) \ FOLIO_CLEAR_FLAG(lname, FOLIO_##policy) \ static __always_inline void ClearPage##uname(struct page *page) \ -{ clear_bit(PG_##lname, &policy(page, 1)->flags); } +{ \ + clear_bit(PG_##lname, &policy(page, 1)->flags); \ + dept_page_clear_bit(page, PG_##lname); \ +} =20 #define __SETPAGEFLAG(uname, lname, policy) \ __FOLIO_SET_FLAG(lname, FOLIO_##policy) \ static __always_inline void __SetPage##uname(struct page *page) \ -{ __set_bit(PG_##lname, &policy(page, 1)->flags); } +{ \ + __set_bit(PG_##lname, &policy(page, 1)->flags); \ + dept_page_set_bit(page, PG_##lname); \ +} =20 #define __CLEARPAGEFLAG(uname, lname, policy) \ __FOLIO_CLEAR_FLAG(lname, FOLIO_##policy) \ static __always_inline void __ClearPage##uname(struct page *page) \ -{ __clear_bit(PG_##lname, &policy(page, 1)->flags); } +{ \ + __clear_bit(PG_##lname, &policy(page, 1)->flags); \ + dept_page_clear_bit(page, PG_##lname); \ +} =20 #define TESTSETFLAG(uname, lname, policy) \ FOLIO_TEST_SET_FLAG(lname, FOLIO_##policy) \ static __always_inline int TestSetPage##uname(struct page *page) \ -{ return test_and_set_bit(PG_##lname, &policy(page, 1)->flags); } +{ \ + bool ret =3D test_and_set_bit(PG_##lname, &policy(page, 1)->flags);\ + if (!ret) \ + dept_page_set_bit(page, PG_##lname); \ + return ret; \ +} =20 #define TESTCLEARFLAG(uname, lname, policy) \ FOLIO_TEST_CLEAR_FLAG(lname, FOLIO_##policy) \ static __always_inline int TestClearPage##uname(struct page *page) \ -{ return test_and_clear_bit(PG_##lname, &policy(page, 1)->flags); } +{ \ + bool ret =3D test_and_clear_bit(PG_##lname, &policy(page, 1)->flags);\ + if (ret) \ + dept_page_clear_bit(page, PG_##lname); \ + return ret; \ +} =20 #define PAGEFLAG(uname, lname, policy) \ TESTPAGEFLAG(uname, lname, policy) \ diff --git a/include/linux/pagemap.h b/include/linux/pagemap.h index 26baa78f1ca7..c4c9742c7a29 100644 --- a/include/linux/pagemap.h +++ b/include/linux/pagemap.h @@ -1097,7 +1097,12 @@ void folio_unlock(struct folio *folio); */ static inline bool folio_trylock(struct folio *folio) { - return likely(!test_and_set_bit_lock(PG_locked, folio_flags(folio, 0))); + bool ret =3D !test_and_set_bit_lock(PG_locked, folio_flags(folio, 0)); + + if (ret) + dept_page_set_bit(&folio->page, PG_locked); + + return likely(ret); } =20 /* @@ -1133,6 +1138,15 @@ static inline bool trylock_page(struct page *page) static inline void folio_lock(struct folio *folio) { might_sleep(); + /* + * dept_page_wait_on_bit() will be called if __folio_lock() goes + * through a real wait path. However, for better job to detect + * *potential* deadlocks, let's assume that folio_lock() always + * goes through wait so that dept can take into account all the + * potential cases. + */ + dept_page_wait_on_bit(&folio->page, PG_locked); + if (!folio_trylock(folio)) __folio_lock(folio); } diff --git a/mm/filemap.c b/mm/filemap.c index 7b90cbeb4a1a..cab03b41add2 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -48,6 +48,7 @@ #include #include #include +#include #include #include #include "internal.h" @@ -1145,6 +1146,7 @@ static int wake_page_function(wait_queue_entry_t *wai= t, unsigned mode, int sync, if (flags & WQ_FLAG_CUSTOM) { if (test_and_set_bit(key->bit_nr, &key->folio->flags)) return -1; + dept_page_set_bit(&key->folio->page, key->bit_nr); flags |=3D WQ_FLAG_DONE; } } @@ -1228,6 +1230,7 @@ static inline bool folio_trylock_flag(struct folio *f= olio, int bit_nr, if (wait->flags & WQ_FLAG_EXCLUSIVE) { if (test_and_set_bit(bit_nr, &folio->flags)) return false; + dept_page_set_bit(&folio->page, bit_nr); } else if (test_bit(bit_nr, &folio->flags)) return false; =20 @@ -1235,6 +1238,9 @@ static inline bool folio_trylock_flag(struct folio *f= olio, int bit_nr, return true; } =20 +struct dept_map __maybe_unused pg_locked_map =3D DEPT_MAP_INITIALIZER(pg_l= ocked_map, NULL); +EXPORT_SYMBOL(pg_locked_map); + static inline int folio_wait_bit_common(struct folio *folio, int bit_nr, int state, enum behavior behavior) { @@ -1246,6 +1252,8 @@ static inline int folio_wait_bit_common(struct folio = *folio, int bit_nr, unsigned long pflags; bool in_thrashing; =20 + dept_page_wait_on_bit(&folio->page, bit_nr); + if (bit_nr =3D=3D PG_locked && !folio_test_uptodate(folio) && folio_test_workingset(folio)) { delayacct_thrashing_start(&in_thrashing); @@ -1339,6 +1347,23 @@ static inline int folio_wait_bit_common(struct folio= *folio, int bit_nr, break; } =20 + /* + * dept_page_set_bit() might have been called already in + * folio_trylock_flag(), wake_page_function() or somewhere. + * However, call it again to reset the wgen of dept to ensure + * dept_page_wait_on_bit() is called prior to + * dept_page_set_bit(). + * + * Remind dept considers all the waits between + * dept_page_set_bit() and dept_page_clear_bit() as potential + * event disturbers. Ensure the correct sequence so that dept + * can make correct decisions: + * + * wait -> acquire(set bit) -> release(clear bit) + */ + if (wait->flags & WQ_FLAG_DONE) + dept_page_set_bit(&folio->page, bit_nr); + /* * If a signal happened, this 'finish_wait()' may remove the last * waiter from the wait-queues, but the folio waiters bit will remain @@ -1496,6 +1521,7 @@ void folio_unlock(struct folio *folio) BUILD_BUG_ON(PG_waiters !=3D 7); BUILD_BUG_ON(PG_locked > 7); VM_BUG_ON_FOLIO(!folio_test_locked(folio), folio); + dept_page_clear_bit(&folio->page, PG_locked); if (folio_xor_flags_has_waiters(folio, 1 << PG_locked)) folio_wake_bit(folio, PG_locked); } diff --git a/mm/mm_init.c b/mm/mm_init.c index 327764ca0ee4..39cf0bc355ba 100644 --- a/mm/mm_init.c +++ b/mm/mm_init.c @@ -31,6 +31,7 @@ #include #include #include +#include #include "internal.h" #include "slab.h" #include "shuffle.h" @@ -586,6 +587,7 @@ void __meminit __init_single_page(struct page *page, un= signed long pfn, atomic_set(&page->_mapcount, -1); page_cpupid_reset_last(page); page_kasan_tag_reset(page); + dept_ext_wgen_init(&page->pg_locked_wgen); =20 INIT_LIST_HEAD(&page->lru); #ifdef WANT_PAGE_VIRTUAL --=20 2.17.1 From nobody Fri Dec 19 15:51:27 2025 Received: from invmail4.hynix.com (exvmail4.hynix.com [166.125.252.92]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 0A35C27978F; Mon, 19 May 2025 09:18:57 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=166.125.252.92 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747646343; cv=none; b=UByRoWvLk3FNL1mkvQ8czrvNuRFfn/JnRpBgOg1WnTkPQOXMNqwt77fPtabPK+wCtrlr5Du8zeKvHAX2wiXvoqjdbbhnehZIYClBnYS6SvXtm9nyM5HdrWwNOjpWaHNBOOZsVVEivNmAlbLQ4xsD3SewunDJhYchBP2o91WYUPI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747646343; c=relaxed/simple; bh=xIkkCdiOJcy9hW8Y2iXba7Lek3wWMYXG6Tsggjp1lwA=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References; b=UiTr2w0vnqXS4T+fkqekX6OzD9qifsQe0F77r36Hio/DdDcWnfCJXKVpSzVLxrASINWrqhrMss3x2Ge8DYIC8ALAmjcZo84S47EYhl0AfaBnz6oMSIvYwvgRGu/zFQ7Fc6tDz0jxfZe2cKrOSjpKX3L/v3khQs+mp4Tn5B93GbA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com; spf=pass smtp.mailfrom=sk.com; arc=none smtp.client-ip=166.125.252.92 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=sk.com X-AuditID: a67dfc5b-681ff7000002311f-6d-682af76fcbf6 From: Byungchul Park To: linux-kernel@vger.kernel.org Cc: kernel_team@skhynix.com, torvalds@linux-foundation.org, damien.lemoal@opensource.wdc.com, linux-ide@vger.kernel.org, adilger.kernel@dilger.ca, linux-ext4@vger.kernel.org, mingo@redhat.com, peterz@infradead.org, will@kernel.org, tglx@linutronix.de, rostedt@goodmis.org, joel@joelfernandes.org, sashal@kernel.org, daniel.vetter@ffwll.ch, duyuyang@gmail.com, johannes.berg@intel.com, tj@kernel.org, tytso@mit.edu, willy@infradead.org, david@fromorbit.com, amir73il@gmail.com, gregkh@linuxfoundation.org, kernel-team@lge.com, linux-mm@kvack.org, akpm@linux-foundation.org, mhocko@kernel.org, minchan@kernel.org, hannes@cmpxchg.org, vdavydov.dev@gmail.com, sj@kernel.org, jglisse@redhat.com, dennis@kernel.org, cl@linux.com, penberg@kernel.org, rientjes@google.com, vbabka@suse.cz, ngupta@vflare.org, linux-block@vger.kernel.org, josef@toxicpanda.com, linux-fsdevel@vger.kernel.org, jack@suse.cz, jlayton@kernel.org, dan.j.williams@intel.com, hch@infradead.org, djwong@kernel.org, dri-devel@lists.freedesktop.org, rodrigosiqueiramelo@gmail.com, melissa.srw@gmail.com, hamohammed.sa@gmail.com, harry.yoo@oracle.com, chris.p.wilson@intel.com, gwan-gyeong.mun@intel.com, max.byungchul.park@gmail.com, boqun.feng@gmail.com, longman@redhat.com, yskelg@gmail.com, yunseong.kim@ericsson.com, yeoreum.yun@arm.com, netdev@vger.kernel.org, matthew.brost@intel.com, her0gyugyu@gmail.com Subject: [PATCH v16 26/42] dept: print staged wait's stacktrace on report Date: Mon, 19 May 2025 18:18:10 +0900 Message-Id: <20250519091826.19752-27-byungchul@sk.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20250519091826.19752-1-byungchul@sk.com> References: <20250519091826.19752-1-byungchul@sk.com> X-Brightmail-Tracker: H4sIAAAAAAAAAzWSfUzMcRzHfX/PXc5+O08/DxtuDJk4U/vMMH/xs7EZ/sGGm35zt564cso0 pTtyKcnqSHI97LQ6yl2jcKR0VB6ic04qdWvNcdXk7qiO3GX++ey1z3vv1+efD4NLWsj5jDIh WVAlyOOklIgQDU0vXZ34M0KxdlgD4PNmEVBcY6Kg4041AlNdBgbulm3wwe9BMPHqDQ76gg4E pf09ONTZehFYK89S0DkwA+y+EQpaC7IpyCyvoeDttwAG3YX5GFSbd8Jn4yAB7XllGOjdFFzX Z2LB8QWDMWMVDcb0ZeCqLKIh0C+D1l4HCdauVXCtpJuCR9ZWAmz1Lgw6HxRT0GuaJKHd9oIA f+4C6LicQ8Lt4TIKvvmNOBh9IzS8azRgYDPMgVpNUHjuxx8Snuc0YnCu4i4G9o8PETzO6sPA bHJQ0OzzYGAxF+AwfqsFgSt3iAbtxTEarmfkIsjWFhKg6Y6CiV/Byze8Msi4WUvA7d8OtGUT byoxIb7ZM4LzGstJftz3nuKtfgPBt5VxfENRD81rHnfRvMF8grdURvDlj9wYXzrqI3lz1QWK N4/m07xuyI7xw69f07sW7hdtjBHilGpBtWbzYZHi86/IY4FNKYVXHVQ6cst0iGE4dj1nz0rT obAp/DRup0NMscs5p3MMD/EsdjFnyRkkdUjE4KwjnPtw4yMKBTPZ7ZxrsI0IMcEu4y6PGYmQ U8xGcw23kv85F3HVtY1TnrDguiu7eaoqYaM4e3UJEXJyrD6Msw5r8X+FedzTSieRh8QGNK0K SZQJ6ni5Mm59pCI1QZkSeSQx3oyCz2VMCxyoR6Mde5oQyyDpdHGtdaVCQsrVSanxTYhjcOks cZVlhUIijpGnnhJUiYdUJ+KEpCa0gCGkc8Xr/CdjJOxRebIQKwjHBNX/FGPC5qejK09o0IVf io0+/rx8zTr1+dJ7rp179x3cm/WmDTIPomfftV/72rVHd79MexfrWeEpKO+Jygj3Gtys7X6y d1AWrt41MbD7ZdHSyTt5WvvDDQONzmKVtK+e5KIrLJ2srCXAjM6uOW3Z+jOtv6d4x+TWM0sk em+Ks8xEFr5/1ZwvlRJJCrksAlclyf8CoBHhQ1gDAAA= X-Brightmail-Tracker: H4sIAAAAAAAAAzWSbUxTZxTHfe47ZddcK5k3srmlppmigiRUz+JGtpjo4+LbN+O+jCo3tgKt aSmKiQlIFYVBxKSAL3QVTCVtFXarUdFqhYgWYkWpVRgvSgyxAmIcrQLVrbjsy8kv55/zO1/+ HKlsphdzekOhZDJo81WMglJsXVe2yvg+Tbe6rWIpRKeOUXC2xcNAzyU3As/lUgIidzfC09g4 gtkHD0mos/UgOPdikITLnUMIfM2HGeh9OR9C0UkGArZKBsqaWhh4NBYnYKD2JAFueQsMO0cp 6D7RSEBdhIEzdWVEYrwiYNrpYsFZooaR5tMsxF9kQmAoTENHQ4AGX/8KOGUfYOCmL0BB57UR AnrbzjIw5PmHhu7O+xTEqlOhp6aKhotvGhkYizlJcEYnWXjsdxDQ6fgSWq0J69G/P9Fwr8pP wNHzfxIQ6ruB4Nax5wTInjADHdFxAryyjYSZC3cRjFRPsHDk92kWzpRWI6g8UkuBdUADsx8S nxumMqH0j1YKLn4Mo5+yscfuQbhjfJLEVu9+PBN9wmBfzEHhrkYRXz89yGLrrX4WO2QL9jan 4aabEQKfexelsew6zmD53UkWV0yECPwmGGS3f/2r4odcKV9fJJkysnMUuuEP6fviPx6orQ8z JSiSWYGSOFHIEv+aCbFzzAjfic+eTZNznCJ8K3qrRukKpOBIIZwsPm3oQ3PBQmGTODLaRc0x JajFmmlngjmOF9aI1y8U/uf8RnS3+j97khLr/sqOz6dKQSOG3HbqBFI40DwXStEbigq0+nxN ujlPV2zQH0jfbSyQUaI/zkPxmmtoqndjOxI4pPqCb/Ut1ylpbZG5uKAdiRypSuFd3mU6JZ+r LT4omYy/mSz5krkdpXKUahH/yw4pRyns0RZKeZK0TzL9nxJc0uISZKS/X2JNNt97vEF/iTbi 25p40rpByy5ejsiGzV/ZlntTr+TyeU1tszumdCst6p/XB5bVl9Pr+UOvx67Oy2Da9ua8dde7 Xqvtlfbhj6uS8ZLsterG9xMLNNuO70zh/d2x8g2n+g4Hs7LuUJ8iFPcqzbXJL5cLLVEbEQgG 71tUlFmnzUwjTWbtv94ReEw7AwAA X-CFilter-Loop: Reflected Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Currently, print nothing about what event wakes up in report. However, it makes hard to interpret dept's report. Make it print wait's stacktrace that the event wakes up. Signed-off-by: Byungchul Park --- include/linux/dept.h | 5 ++++ include/linux/sched.h | 2 ++ kernel/dependency/dept.c | 59 ++++++++++++++++++++++++++++++++++------ 3 files changed, 57 insertions(+), 9 deletions(-) diff --git a/include/linux/dept.h b/include/linux/dept.h index 236e4f06e5c8..b6dc4ff19537 100644 --- a/include/linux/dept.h +++ b/include/linux/dept.h @@ -227,6 +227,11 @@ struct dept_ecxt { */ unsigned long event_ip; struct dept_stack *event_stack; + + /* + * wait that this event ttwu + */ + struct dept_stack *ewait_stack; }; }; }; diff --git a/include/linux/sched.h b/include/linux/sched.h index 541ddacdc3d0..43927e61921b 100644 --- a/include/linux/sched.h +++ b/include/linux/sched.h @@ -865,6 +865,7 @@ struct dept_task { const char *stage_w_fn; unsigned long stage_ip; bool stage_timeout; + struct dept_stack *stage_wait_stack; arch_spinlock_t stage_lock; =20 /* @@ -906,6 +907,7 @@ struct dept_task { .stage_w_fn =3D NULL, \ .stage_ip =3D 0UL, \ .stage_timeout =3D false, \ + .stage_wait_stack =3D NULL, \ .stage_lock =3D (arch_spinlock_t)__ARCH_SPIN_LOCK_UNLOCKED,\ .missing_ecxt =3D 0, \ .hardirqs_enabled =3D false, \ diff --git a/kernel/dependency/dept.c b/kernel/dependency/dept.c index 79357a3a03bb..dc3effabfab4 100644 --- a/kernel/dependency/dept.c +++ b/kernel/dependency/dept.c @@ -523,6 +523,7 @@ static void initialize_ecxt(struct dept_ecxt *e) e->enirqf =3D 0UL; e->event_ip =3D 0UL; e->event_stack =3D NULL; + e->ewait_stack =3D NULL; } SET_CONSTRUCTOR(ecxt, initialize_ecxt); =20 @@ -578,6 +579,8 @@ static void destroy_ecxt(struct dept_ecxt *e) put_stack(e->ecxt_stack); if (e->event_stack) put_stack(e->event_stack); + if (e->ewait_stack) + put_stack(e->ewait_stack); } SET_DESTRUCTOR(ecxt, destroy_ecxt); =20 @@ -794,6 +797,11 @@ static void print_dep(struct dept_dep *d) =20 pr_warn("[E] %s(%s:%d):\n", e_fn, fc_n, fc->sub_id); print_ip_stack(e->event_ip, e->event_stack); + + if (valid_stack(e->ewait_stack)) { + pr_warn("(wait to wake up)\n"); + print_ip_stack(0, e->ewait_stack); + } } =20 if (!irqf) { @@ -807,6 +815,11 @@ static void print_dep(struct dept_dep *d) =20 pr_warn("[E] %s(%s:%d):\n", e_fn, fc_n, fc->sub_id); print_ip_stack(e->event_ip, e->event_stack); + + if (valid_stack(e->ewait_stack)) { + pr_warn("(wait to wake up)\n"); + print_ip_stack(0, e->ewait_stack); + } } } =20 @@ -1657,7 +1670,8 @@ static void add_wait(struct dept_class *c, unsigned l= ong ip, =20 static struct dept_ecxt_held *add_ecxt(struct dept_map *m, struct dept_class *c, unsigned long ip, const char *c_fn, - const char *e_fn, int sub_l) + const char *e_fn, int sub_l, + struct dept_stack *ewait_stack) { struct dept_task *dt =3D dept_task(); struct dept_ecxt_held *eh; @@ -1691,6 +1705,7 @@ static struct dept_ecxt_held *add_ecxt(struct dept_ma= p *m, e->class =3D get_class(c); e->ecxt_ip =3D ip; e->ecxt_stack =3D ip ? get_current_stack() : NULL; + e->ewait_stack =3D ewait_stack ? get_stack(ewait_stack) : NULL; e->event_fn =3D e_fn; e->ecxt_fn =3D c_fn; =20 @@ -1797,7 +1812,7 @@ static int find_hist_pos(unsigned int wg) =20 static void do_event(struct dept_map *m, struct dept_map *real_m, struct dept_class *c, unsigned int wg, unsigned long ip, - const char *e_fn) + const char *e_fn, struct dept_stack *ewait_stack) { struct dept_task *dt =3D dept_task(); struct dept_wait_hist *wh; @@ -1825,7 +1840,7 @@ static void do_event(struct dept_map *m, struct dept_= map *real_m, */ if (find_ecxt_pos(real_m, c, false) !=3D -1) return; - eh =3D add_ecxt(m, c, 0UL, NULL, e_fn, 0); + eh =3D add_ecxt(m, c, 0UL, NULL, e_fn, 0, ewait_stack); =20 if (!eh) return; @@ -2360,7 +2375,8 @@ static void __dept_wait(struct dept_map *m, unsigned = long w_f, */ static void __dept_event(struct dept_map *m, struct dept_map *real_m, unsigned long e_f, unsigned long ip, const char *e_fn, - bool sched_map, unsigned int wg) + bool sched_map, unsigned int wg, + struct dept_stack *ewait_stack) { struct dept_class *c; struct dept_key *k; @@ -2382,7 +2398,7 @@ static void __dept_event(struct dept_map *m, struct d= ept_map *real_m, c =3D check_new_class(&m->map_key, k, sub_id(m, e), m->name, sched_map); =20 if (c) - do_event(m, real_m, c, wg, ip, e_fn); + do_event(m, real_m, c, wg, ip, e_fn, ewait_stack); } =20 void dept_wait(struct dept_map *m, unsigned long w_f, @@ -2498,6 +2514,9 @@ static void __dept_clean_stage(struct dept_task *dt) dt->stage_w_fn =3D NULL; dt->stage_ip =3D 0UL; dt->stage_timeout =3D false; + if (dt->stage_wait_stack) + put_stack(dt->stage_wait_stack); + dt->stage_wait_stack =3D NULL; } =20 void dept_clean_stage(void) @@ -2561,6 +2580,14 @@ void dept_request_event_wait_commit(void) =20 wg =3D next_wgen(); WRITE_ONCE(dt->stage_m.wgen, wg); + + /* + * __schedule() can be hit multiple times between + * dept_stage_wait() and dept_clean_stage(). In that case, + * keep the first stacktrace only. That's enough. + */ + if (!dt->stage_wait_stack) + dt->stage_wait_stack =3D get_current_stack(); arch_spin_unlock(&dt->stage_lock); =20 __dept_wait(&dt->stage_m, 1UL, ip, w_fn, 0, true, sched_map, timeout); @@ -2579,6 +2606,7 @@ void dept_ttwu_stage_wait(struct task_struct *request= or, unsigned long ip) struct dept_map m; struct dept_map *real_m; bool sched_map; + struct dept_stack *ewait_stack; =20 if (unlikely(!dept_working())) return; @@ -2597,6 +2625,10 @@ void dept_ttwu_stage_wait(struct task_struct *reques= tor, unsigned long ip) m =3D dt_req->stage_m; sched_map =3D dt_req->stage_sched_map; real_m =3D dt_req->stage_real_m; + ewait_stack =3D dt_req->stage_wait_stack; + if (ewait_stack) + get_stack(ewait_stack); + __dept_clean_stage(dt_req); arch_spin_unlock(&dt_req->stage_lock); =20 @@ -2607,8 +2639,12 @@ void dept_ttwu_stage_wait(struct task_struct *reques= tor, unsigned long ip) if (!m.keys) goto exit; =20 - __dept_event(&m, real_m, 1UL, ip, "try_to_wake_up", sched_map, m.wgen); + __dept_event(&m, real_m, 1UL, ip, "try_to_wake_up", sched_map, + m.wgen, ewait_stack); exit: + if (ewait_stack) + put_stack(ewait_stack); + dept_exit(flags); } =20 @@ -2688,7 +2724,7 @@ void dept_map_ecxt_modify(struct dept_map *m, unsigne= d long e_f, k =3D m->keys ?: &m->map_key; c =3D check_new_class(&m->map_key, k, sub_id(m, new_e), m->name, false); =20 - if (c && add_ecxt(m, c, new_ip, new_c_fn, new_e_fn, new_sub_l)) + if (c && add_ecxt(m, c, new_ip, new_c_fn, new_e_fn, new_sub_l, NULL)) goto exit; =20 /* @@ -2740,7 +2776,7 @@ void dept_ecxt_enter(struct dept_map *m, unsigned lon= g e_f, unsigned long ip, k =3D m->keys ?: &m->map_key; c =3D check_new_class(&m->map_key, k, sub_id(m, e), m->name, false); =20 - if (c && add_ecxt(m, c, ip, c_fn, e_fn, sub_l)) + if (c && add_ecxt(m, c, ip, c_fn, e_fn, sub_l, NULL)) goto exit; missing_ecxt: dt->missing_ecxt++; @@ -2840,7 +2876,7 @@ void dept_event(struct dept_map *m, unsigned long e_f, =20 flags =3D dept_enter(); =20 - __dept_event(m, m, e_f, ip, e_fn, false, READ_ONCE(*wg_p)); + __dept_event(m, m, e_f, ip, e_fn, false, READ_ONCE(*wg_p), NULL); =20 /* * Keep the map diabled until the next sleep. @@ -2912,6 +2948,11 @@ void dept_task_exit(struct task_struct *t) dt->stack =3D NULL; } =20 + if (dt->stage_wait_stack) { + put_stack(dt->stage_wait_stack); + dt->stage_wait_stack =3D NULL; + } + for (i =3D 0; i < dt->ecxt_held_pos; i++) { if (dt->ecxt_held[i].class) { put_class(dt->ecxt_held[i].class); --=20 2.17.1 From nobody Fri Dec 19 15:51:27 2025 Received: from invmail4.hynix.com (exvmail4.hynix.com [166.125.252.92]) by smtp.subspace.kernel.org (Postfix) with ESMTP id D270927A116; Mon, 19 May 2025 09:18:58 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=166.125.252.92 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747646343; cv=none; b=Hhc+8bsPDa4Z8hthsgfZYkOzB877nuBEqB+sjYLNNoO0Jp76Hw/H3+/m30s30XkHXdxXUvwJPdCsbO9fCrlcJs+OIrsWF1R+2xDLpfEjtMLkFLt0sPF9aNGVP6Ji/ZqCcVYLHLhLV/egIiwmcyIZ/pmJ+zHZW0OA56p6H0Nh66Q= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747646343; c=relaxed/simple; bh=XT7wM+AY+fy9b7R9FSPBT0zn1xc+Ht0DfCIhlHeaS6U=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References; b=dP9BlTGRG7o/L/eW6ljnlZMbXquGRkedEFVyXahUhMdrZPq0JeNN7Y0dcbu14g0Tz0LoXGkzhUwUM94N0BHrUwSxVK9a7zIUGzSyrnNxEkC7rhyxRJ2nufd7Of2+NloMObip6JbURc5l2IPKozgk37uW2YikV1ltVQ2Jdj0jJdE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com; spf=pass smtp.mailfrom=sk.com; arc=none smtp.client-ip=166.125.252.92 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=sk.com X-AuditID: a67dfc5b-681ff7000002311f-7c-682af76f65b6 From: Byungchul Park To: linux-kernel@vger.kernel.org Cc: kernel_team@skhynix.com, torvalds@linux-foundation.org, damien.lemoal@opensource.wdc.com, linux-ide@vger.kernel.org, adilger.kernel@dilger.ca, linux-ext4@vger.kernel.org, mingo@redhat.com, peterz@infradead.org, will@kernel.org, tglx@linutronix.de, rostedt@goodmis.org, joel@joelfernandes.org, sashal@kernel.org, daniel.vetter@ffwll.ch, duyuyang@gmail.com, johannes.berg@intel.com, tj@kernel.org, tytso@mit.edu, willy@infradead.org, david@fromorbit.com, amir73il@gmail.com, gregkh@linuxfoundation.org, kernel-team@lge.com, linux-mm@kvack.org, akpm@linux-foundation.org, mhocko@kernel.org, minchan@kernel.org, hannes@cmpxchg.org, vdavydov.dev@gmail.com, sj@kernel.org, jglisse@redhat.com, dennis@kernel.org, cl@linux.com, penberg@kernel.org, rientjes@google.com, vbabka@suse.cz, ngupta@vflare.org, linux-block@vger.kernel.org, josef@toxicpanda.com, linux-fsdevel@vger.kernel.org, jack@suse.cz, jlayton@kernel.org, dan.j.williams@intel.com, hch@infradead.org, djwong@kernel.org, dri-devel@lists.freedesktop.org, rodrigosiqueiramelo@gmail.com, melissa.srw@gmail.com, hamohammed.sa@gmail.com, harry.yoo@oracle.com, chris.p.wilson@intel.com, gwan-gyeong.mun@intel.com, max.byungchul.park@gmail.com, boqun.feng@gmail.com, longman@redhat.com, yskelg@gmail.com, yunseong.kim@ericsson.com, yeoreum.yun@arm.com, netdev@vger.kernel.org, matthew.brost@intel.com, her0gyugyu@gmail.com Subject: [PATCH v16 27/42] locking/lockdep: prevent various lockdep assertions when lockdep_off()'ed Date: Mon, 19 May 2025 18:18:11 +0900 Message-Id: <20250519091826.19752-28-byungchul@sk.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20250519091826.19752-1-byungchul@sk.com> References: <20250519091826.19752-1-byungchul@sk.com> X-Brightmail-Tracker: H4sIAAAAAAAAAzWSa0yTdxTG/b93Oqqv79x8FRJNIyiYsbGoOZnTqDHhnxijG9nM5get8GZt xi2tFtEsoVqMA4uKqSi3Fhzl0jJL6wdAS5BLBzM4mB0CARTizJBiM6RVpNO1GL+cPHme8/zO l8ORQje9llNnHZc0WcoMBSOjZLPR1Z/kvExUfdathcD8eQoqbtoZGPjVhsB+S0/AdE8KPAz6 ECz2/0FCqWkAQfXkOAm3PBMI3PVnGHjwZDl4A34G+kxFDJy9cZOBwZkQAWNXSwiwOffDI+tT Cu5dqiGgdJqB8tKzRHj8Q8CCtZEFa34cTNWXsRCaTIa+iSEa3KOb4XrVGAN33H0UeFqmCHjQ VsHAhP0tDfc8vRQEi2Ng4LKRhqbnNQzMBK0kWAN+Fv7ssBDgsXwMDkMYeO7FGxp+M3YQcO6X ZgK8I7cRtJ9/TIDTPsRAV8BHgMtpIuF1XQ+CqeJZFgouLLBQri9GUFRwlQLD2FZYfBW+XDmf DHqzg4Km/4bQrh3YXmVHuMvnJ7HBlYtfB/5isDtoofDvNSJuLRtnsaF9lMUW5wnsqk/EN+5M E7h6LkBjZ+PPDHbOlbC4cNZL4Of377MHY7+XfZkuZah1kubTnUdlqh5/fM7IypNWv5HJR9YV hSiKE/kt4mybni5E3JJuajgVsRl+ozg8vEBG9Cp+vegyPg2vyDiSH/pAfFg5giLBh3y62DdT QEQ0xceJdd6/lzhyfpv4yhvzDr9OtDk6ljhRYXu0qGupKvBbRa+tioowRd4cJRa31tHvCmvE u/XD1CUkt6BljUhQZ+kyleqMLUmqvCz1yaS07EwnCv+W9afQ4RY0N5DaiXgOKaLlDneCSqCV Om1eZicSOVKxSt7o2qQS5OnKvFOSJvuI5kSGpO1EMRylWC3/PJibLvA/KI9LP0pSjqR5nxJc 1Np8tDthU3NZdkD7RKiJX1nx9faL374wdwfxZfOxDqPnC1fvkdjYtJLFfwftV3oM+1MPjGe3 6Jr3JTwKBR2mwdprTb41B476os3jG/otzx7n7uj9qq3yo28Ce8x7OFsKCjV8d4Y8ZC+f36sT TnvzTMKhNL7Wo1s9XNt/WtGqn5xOyVmuoLQqZXIiqdEq/wdVo8PjVwMAAA== X-Brightmail-Tracker: H4sIAAAAAAAAAzWSf0jTeRzG+3x/b3erb0vsWxLFoEwjS8p4R4cW3NWHoAgKgvojR35pI52y qZdRoTftUtOzyEaZNn80TWeuzcrKDXGprbid5fLXmaX0y9NarG2XP65rC/rn4eF5eL+ef94c Ka+nl3JqTaao1ShTFYyUku7eol+b/m+san1XeQwE/GcouNJiZqD3RhMCc2seARNdO2AgOIVg 9s+/SDCU9yKoHntOQmv3KAJ7w28M9L2aD56AlwFXeTED+toWBp5MzhEwcvE8AU3WXfDC9IaC x2U1BBgmGKgw6ImQvCNg2tTIgil3JYw3XGZhbiweXKP9NDgrXTTYh9fApaoRBtrtLgq628YJ 6Lt3hYFR8/80PO5+SEGwNAp6z5XQ0PyhhoHJoIkEU8DLwtMOIwHdxkiw5Ieopz99oaGnpIOA 03U3CfAM3UfgOPOSAKu5nwFnYIoAm7WchJn6LgTjpe9ZKDg7zUJFXimC4oKLFOSPJMDs59By pT8e8q5aKGj+rx9tTcTmKjPCzikvifNtv+KZwDMG24NGCj+qEfDdy89ZnO8YZrHRmoVtDbG4 tn2CwNW+AI2tjYUMtvrOs7jovYfAH9xuds+yA9KfUsRUdbaoXZeYLFV1eVdlDC08ZvKWMLnI tKAIcZzAbxSarx8vQhKO4aOFwcFpMuwj+BWCreQNXYSkHMn3/yAMVA6hcLGITxFckwVE2FP8 SqHe85oOc2T8JuGzJyocC/xyocnS8Y0jCcXDxc5vp3I+QfA0VVFlSGpE8xpRhFqTnaZUpybE 6Y6qcjTqY3GH09OsKPQ+ppNz59qQv29HJ+I5pPhRZrHHqOS0MluXk9aJBI5URMgabatVclmK Mue4qE0/pM1KFXWdKIqjFItlO/eLyXL+iDJTPCqKGaL2e0twkqW5CCI/bQje8vmXrB2ovp0+ 6FYeLGv4pT1Wc2P3/X+cQ75HSXfMPbprM9QfFmdFouOLbJ17hVvVI3PsM2j00TbptgeFT/QZ hbrsJRsLObzmmu/FljZHpn/MsP1A3KmUvdExSZH+C5Tz5onew3/XXdgsefv7vaTW/cm1WZbp lp9nJeijgtKplPGxpFan/Ar3k/K5OgMAAA== X-CFilter-Loop: Reflected Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" lockdep provides APIs for assertion only if lockdep is enabled at the moment asserting to avoid unnecessary confusion, using the following condition, debug_locks && !this_cpu_read(lockdep_recursion). However, lockdep_{off,on}() are also used for disabling and enabling lockdep for a simular purpose. Add !lockdep_recursing(current) that is updated by lockdep_{off,on}() to the condition so that the assertions are aware of !__lockdep_enabled if lockdep_off()'ed. Signed-off-by: Byungchul Park --- include/linux/lockdep.h | 3 ++- kernel/locking/lockdep.c | 10 ++++++++++ 2 files changed, 12 insertions(+), 1 deletion(-) diff --git a/include/linux/lockdep.h b/include/linux/lockdep.h index ef03d8808c10..c83fe95199db 100644 --- a/include/linux/lockdep.h +++ b/include/linux/lockdep.h @@ -303,6 +303,7 @@ extern void lock_unpin_lock(struct lockdep_map *lock, s= truct pin_cookie); lockdep_assert_once(!current->lockdep_depth) =20 #define lockdep_recursing(tsk) ((tsk)->lockdep_recursion) +extern bool lockdep_recursing_current(void); =20 #define lockdep_pin_lock(l) lock_pin_lock(&(l)->dep_map) #define lockdep_repin_lock(l,c) lock_repin_lock(&(l)->dep_map, (c)) @@ -630,7 +631,7 @@ DECLARE_PER_CPU(int, hardirqs_enabled); DECLARE_PER_CPU(int, hardirq_context); DECLARE_PER_CPU(unsigned int, lockdep_recursion); =20 -#define __lockdep_enabled (debug_locks && !this_cpu_read(lockdep_recursion= )) +#define __lockdep_enabled (debug_locks && !this_cpu_read(lockdep_recursion= ) && !lockdep_recursing_current()) =20 #define lockdep_assert_irqs_enabled() \ do { \ diff --git a/kernel/locking/lockdep.c b/kernel/locking/lockdep.c index 6c984a55d5ed..d2805ce250cb 100644 --- a/kernel/locking/lockdep.c +++ b/kernel/locking/lockdep.c @@ -6889,3 +6889,13 @@ void lockdep_rcu_suspicious(const char *file, const = int line, const char *s) warn_rcu_exit(rcu); } EXPORT_SYMBOL_GPL(lockdep_rcu_suspicious); + +/* + * For avoiding header dependency when using (struct task_struct *)current + * and lockdep_recursing() at the same time. + */ +noinstr bool lockdep_recursing_current(void) +{ + return lockdep_recursing(current); +} +EXPORT_SYMBOL_GPL(lockdep_recursing_current); --=20 2.17.1 From nobody Fri Dec 19 15:51:27 2025 Received: from invmail4.hynix.com (exvmail4.hynix.com [166.125.252.92]) by smtp.subspace.kernel.org (Postfix) with ESMTP id DC07E27A121; Mon, 19 May 2025 09:18:58 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=166.125.252.92 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747646344; cv=none; b=RyZT3gMcr+X8gFTgEu0obFzDPka/x5rg+POXsBuEp1A/89rLhJiX2BVkka+vpQlgWpMof3GMn4tLvgBdvS/FnXXBrON3LDWlJNzUnqzZRwGOt3yZ52WlK+hyZYWqHcWEOJzwxUrbJJJ9IJ2YHJDPj7+OAZFA20R2pEbjxnZUeGc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747646344; c=relaxed/simple; bh=faITj/Q/isOD+29VNf4M4bgUiNP5RcjvtJE110dADOo=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References; b=UVQAzeeUB36zAhXjVIb1J1osPa6NB9JIXrZ5IfRbxMROl15GPTRFA318CqZgDjnOitdvXlZ+atV2U312lO++qAppII/JAd/0OcSf0gGVJSMxHmk4HVEah+o/pLIBCBCaEspE1By0KKj+RsS8wHeZRoECo76cVs6Kj9SLhu609zs= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com; spf=pass smtp.mailfrom=sk.com; arc=none smtp.client-ip=166.125.252.92 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=sk.com X-AuditID: a67dfc5b-681ff7000002311f-8b-682af76f766c From: Byungchul Park To: linux-kernel@vger.kernel.org Cc: kernel_team@skhynix.com, torvalds@linux-foundation.org, damien.lemoal@opensource.wdc.com, linux-ide@vger.kernel.org, adilger.kernel@dilger.ca, linux-ext4@vger.kernel.org, mingo@redhat.com, peterz@infradead.org, will@kernel.org, tglx@linutronix.de, rostedt@goodmis.org, joel@joelfernandes.org, sashal@kernel.org, daniel.vetter@ffwll.ch, duyuyang@gmail.com, johannes.berg@intel.com, tj@kernel.org, tytso@mit.edu, willy@infradead.org, david@fromorbit.com, amir73il@gmail.com, gregkh@linuxfoundation.org, kernel-team@lge.com, linux-mm@kvack.org, akpm@linux-foundation.org, mhocko@kernel.org, minchan@kernel.org, hannes@cmpxchg.org, vdavydov.dev@gmail.com, sj@kernel.org, jglisse@redhat.com, dennis@kernel.org, cl@linux.com, penberg@kernel.org, rientjes@google.com, vbabka@suse.cz, ngupta@vflare.org, linux-block@vger.kernel.org, josef@toxicpanda.com, linux-fsdevel@vger.kernel.org, jack@suse.cz, jlayton@kernel.org, dan.j.williams@intel.com, hch@infradead.org, djwong@kernel.org, dri-devel@lists.freedesktop.org, rodrigosiqueiramelo@gmail.com, melissa.srw@gmail.com, hamohammed.sa@gmail.com, harry.yoo@oracle.com, chris.p.wilson@intel.com, gwan-gyeong.mun@intel.com, max.byungchul.park@gmail.com, boqun.feng@gmail.com, longman@redhat.com, yskelg@gmail.com, yunseong.kim@ericsson.com, yeoreum.yun@arm.com, netdev@vger.kernel.org, matthew.brost@intel.com, her0gyugyu@gmail.com Subject: [PATCH v16 28/42] dept: add documentation for dept Date: Mon, 19 May 2025 18:18:12 +0900 Message-Id: <20250519091826.19752-29-byungchul@sk.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20250519091826.19752-1-byungchul@sk.com> References: <20250519091826.19752-1-byungchul@sk.com> X-Brightmail-Tracker: H4sIAAAAAAAAAzWSb0xbZRSHfd9773tvu9XcdESv4KKpIyrGSedYTnQufnDxzRJ1cV+MU2dj r2sDdNhuMIwGupUG6cAxwtjYwMKw1LZztZ0ZKl2QShHRWUctFIFtqETGP8NokQ2nhWxfTn55 Ts5zzocjMOo+LlMwmvbLZpOuQEOUrHJmbcuTRYs5htyB+BpILlSycPqcj0D0cy8C33krhsme F2EwNY3g1k8/M9BQH0XQcm2UgfORMQQh9yECA3/cC7HkHIG+egeBw2fOEfhlahnDyPFjGLyB l+CKa4KF/qOtGBomCZxqOIzT5S8MSy4PD67ybBh3N/KwfE0LfWNxDkLDT8DJ5hECnaE+FiId 4xgGvj5NYMz3Hwf9ke9ZSNVkQbS2moOzs60EplIuBlzJOR4udzkxRJz3gd+WFtpv3Oagt7oL g73tCwyxxDcILlZexRDwxQmEk9MYgoF6Bm629yAYr5nhoeLIEg+nrDUIHBXHWbCN5MGtf9Kb mxa0YP3Ez8LZf+Po+eeor9mHaHh6jqG2YAm9mfyV0FDKydIfWiX6VeMoT20Xh3nqDBygQXcO PdM5iWnLfJKjAc9HhAbmj/G0aiaG6eylS/zOB19XbtXLBcZi2fzUtreVho7Qj1zR9Rg6uBCZ JOWotwlVIYUgiZuluguV+G722BeZlUzER6WhoaXVnCE+LAWrJ7gqpBQYMb5GGmxKrA6vE5+V /AthUoUEgRWzJcff61awStwiDba33fE/JHn9XaseRZoPO8KrXC3mSTFvM7vilMQ6hXRh4rM7 RzwgfeseYo8ilRPd40Fqo6m4UGcs2LzRUGoyHtz4zr7CAEo/mOvD5d0daD66qxuJAtKsVflD jxvUnK7YUlrYjSSB0WSoPMHHDGqVXlf6vmzet8d8oEC2dKMsgdXcr9qUKtGrxb26/XK+LBfJ 5rtdLCgyy9EH+lyt90RTXsXShp4tQhnzdH/C/rKJ3Vb7m/1KZ8b2y47Gwg3XXx3tOWHLbMj9 eCq7F5FDwV3L0hvvfbdd67k9mwjseOHN3cinzfK7Ilv1N+oeKWvfmXAf+fStq7W/v7LDan1t JvrnppKyPfkn9yqgja5/1zJLa/NbF8PPzK3H6EsNazHotDmM2aL7H8di33hcAwAA X-Brightmail-Tracker: H4sIAAAAAAAAAzWSa0hTcRiH+5+7q9VpWR0aUQyjMjKFjBe60wcPQRF9CfpQrTy1lU7Z0loZ eUtsa8NJJnldKtO2VWsz0GpijlZ2tVzzgllJN9MyamemabWkvrz8eH68z/vlZXCZg1zAqDVH Ba1GmaKgJIRk+9q8lWmjsap4sS8exFAhARXXnBR0XHUgcDbmYDB4Nwm6wsMIfj5+ikNpSQeC S29e4tDo70fgbciloPPtTAiIIxS0lxgpyKu9RsGzoQkM+i4UY+Bwb4NXtvcEPCyqwaB0kILy 0jwsMj5iMGaz02DLXgIDDWU0TLxJgPb+IAm+ynYSvL0r4GJVHwW3ve0E+JsGMOi8WUFBv/M3 CQ/99wkIm+XQYTGRcOVLDQVDYRsONnGEhuetVgz81nngyo9YC77/IuGeqRWDgrrrGAR6biFo KXyNgdsZpMAnDmPgcZfgMF5/F8GA+TMNZ86N0VCeY0ZgPHOBgPy+RPj5I3K5MpQAOdUuAq5M BtGmDbyzyol43/AIzud7jvHj4guK94atBP+ghuOby17SfH5LL81b3Rm8pyGWr709iPGXvokk 77afpXj3t2KaN3wOYPyXJ0/oHQt3S9YlCynqTEG7asM+iarJ+4hM/xRAx0P+QSob3atEBhTF cOxqzl4wiv/NFLuU6+4em8rR7GLOY3pPGpCEwdngdK6rsmdqYQ67lnOFfJQBMQzBLuGMX+f8 xVJ2DddVX/fPuYhzuFqnPFER3mv0TXEZm8gFHFVEEZJY0TQ7ilZrMlOV6pTEON0RlV6jPh53 IC3VjSIvZDs1YWlCoc6kNsQySDFD6vIuV8lIZaZOn9qGOAZXREvtnmUqmTRZqT8haNP2ajNS BF0bkjOEYr506y5hn4w9pDwqHBGEdEH7v8WYqAXZqDtaF3XwdLVDPnmjemaMZTzdGJPQU5Ns /jBtz6KSpz9OZj3I0pdNFqXdWd+04/yvnXFii3n13ODF17dyD+LdG+Vba/e/6hiwl7+rk8eY Zg81bFaZcqdL4rMq2NFUi+dwM2PIMZwrHrOlbwknZ6zXFIqzsLBTYdH3fk8Knldd/pilIHQq ZUIsrtUp/wAY/LVmPgMAAA== X-CFilter-Loop: Reflected Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" This document describes the concept and APIs of dept. Signed-off-by: Byungchul Park --- Documentation/dependency/dept.txt | 735 ++++++++++++++++++++++++++ Documentation/dependency/dept_api.txt | 117 ++++ 2 files changed, 852 insertions(+) create mode 100644 Documentation/dependency/dept.txt create mode 100644 Documentation/dependency/dept_api.txt diff --git a/Documentation/dependency/dept.txt b/Documentation/dependency/d= ept.txt new file mode 100644 index 000000000000..5dd358b96734 --- /dev/null +++ b/Documentation/dependency/dept.txt @@ -0,0 +1,735 @@ +DEPT(DEPendency Tracker) +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D + +Started by Byungchul Park + +How lockdep works +----------------- + +Lockdep detects a deadlock by checking lock acquisition order. For +example, a graph to track acquisition order built by lockdep might look +like: + + A -> B - + \ + -> E + / + C -> D - + + where 'A -> B' means that acquisition A is prior to acquisition B + with A still held. + +Lockdep keeps adding each new acquisition order into the graph in +runtime. For example, 'E -> C' will be added when the two locks have +been acquired in the order, E and then C. The graph will look like: + + A -> B - + \ + -> E - + / \ + -> C -> D - \ + / / + \ / + ------------------ + + where 'A -> B' means that acquisition A is prior to acquisition B + with A still held. + +This graph contains a subgraph that demonstrates a loop like: + + -> E - + / \ + -> C -> D - \ + / / + \ / + ------------------ + + where 'A -> B' means that acquisition A is prior to acquisition B + with A still held. + +Lockdep reports it as a deadlock on detection of a loop and stops its +working. + +CONCLUSION + +Lockdep detects a deadlock by checking if a loop has been created after +adding a new acquisition order into the graph. + + +Limitation of lockdep +--------------------- + +Lockdep deals with a deadlock by typical lock e.g. spinlock and mutex, +that are supposed to be released within the acquisition context. However, +when it comes to a deadlock by folio lock that is not supposed to be +released within the acquisition context or other general synchronization +mechanisms, lockdep doesn't work. + +Can lockdep detect the following deadlock? + + context X context Y context Z + + mutex_lock A + folio_lock B + folio_lock B <- DEADLOCK + mutex_lock A <- DEADLOCK + folio_unlock B + folio_unlock B + mutex_unlock A + mutex_unlock A + +No. What about the following? + + context X context Y + + mutex_lock A + mutex_lock A <- DEADLOCK + wait_for_complete B <- DEADLOCK + complete B + mutex_unlock A + mutex_unlock A + +No. + +CONCLUSION + +Lockdep cannot detect a deadlock by folio lock or other general +synchronization mechanisms. + + +What leads a deadlock +--------------------- + +A deadlock occurs when one or multi contexts are waiting for events that +will never happen. For example: + + context X context Y context Z + + | | | + v | | + 1 wait for A v | + . 2 wait for C v + event C . 3 wait for B + event B . + event A + +Event C cannot be triggered because context X is stuck at 1, event B +cannot be triggered because context Y is stuck at 2, and event A cannot +be triggered because context Z is stuck at 3. All the contexts are stuck. +We call this *deadlock*. + +If an event occurrence is a prerequisite to reaching another event, we +call it *dependency*. In this example: + + Event A occurrence is a prerequisite to reaching event C. + Event C occurrence is a prerequisite to reaching event B. + Event B occurrence is a prerequisite to reaching event A. + +In terms of dependency: + + Event C depends on event A. + Event B depends on event C. + Event A depends on event B. + +Dependency graph reflecting this example will look like: + + -> C -> A -> B - + / \ + \ / + ---------------- + + where 'A -> B' means that event A depends on event B. + +A circular dependency exists. Such a circular dependency leads a +deadlock since no waiters can have desired events triggered. + +CONCLUSION + +A circular dependency of events leads a deadlock. + + +Introduce DEPT +-------------- + +DEPT(DEPendency Tracker) tracks wait and event instead of lock +acquisition order so as to recognize the following situation: + + context X context Y context Z + + | | | + v | | + wait for A v | + . wait for C v + event C . wait for B + event B . + event A + +and builds up a dependency graph in runtime that is similar to lockdep. +The graph might look like: + + -> C -> A -> B - + / \ + \ / + ---------------- + + where 'A -> B' means that event A depends on event B. + +DEPT keeps adding each new dependency into the graph in runtime. For +example, 'B -> D' will be added when event D occurrence is a +prerequisite to reaching event B like: + + | + v + wait for D + . + event B + +After the addition, the graph will look like: + + -> D + / + -> C -> A -> B - + / \ + \ / + ---------------- + + where 'A -> B' means that event A depends on event B. + +DEPT is going to report a deadlock on detection of a new loop. + +CONCLUSION + +DEPT works on wait and event so as to theoretically detect all the +potential deadlocks. + + +How DEPT works +-------------- + +Let's take a look how DEPT works with the 1st example in the section +'Limitation of lockdep'. + + context X context Y context Z + + mutex_lock A + folio_lock B + folio_lock B <- DEADLOCK + mutex_lock A <- DEADLOCK + folio_unlock B + folio_unlock B + mutex_unlock A + mutex_unlock A + +Adding comments to describe DEPT's view in terms of wait and event: + + context X context Y context Z + + mutex_lock A + /* wait for A */ + folio_lock B + /* wait for A */ + /* start event A context */ + + folio_lock B + /* wait for B */ <- DEADLOCK + /* start event B context */ + + mutex_lock A + /* wait for A */ <- DEADLOCK + /* start event A context */ + + folio_unlock B + /* event B */ + folio_unlock B + /* event B */ + + mutex_unlock A + /* event A */ + mutex_unlock A + /* event A */ + +Adding more supplementary comments to describe DEPT's view in detail: + + context X context Y context Z + + mutex_lock A + /* might wait for A */ + /* start to take into account event A's context */ + /* 1 */ + folio_lock B + /* might wait for B */ + /* start to take into account event B's context */ + /* 2 */ + + folio_lock B + /* might wait for B */ <- DEADLOCK + /* start to take into account event B's context */ + /* 3 */ + + mutex_lock A + /* might wait for A */ <- DEADLOCK + /* start to take into account + event A's context */ + /* 4 */ + + folio_unlock B + /* event B that's been valid since 2 */ + folio_unlock B + /* event B that's been valid since 3 */ + + mutex_unlock A + /* event A that's been valid since 1 */ + + mutex_unlock A + /* event A that's been valid since 4 */ + +Let's build up dependency graph with this example. Firstly, context X: + + context X + + folio_lock B + /* might wait for B */ + /* start to take into account event B's context */ + /* 2 */ + +There are no events to create dependency. Next, context Y: + + context Y + + mutex_lock A + /* might wait for A */ + /* start to take into account event A's context */ + /* 1 */ + + folio_lock B + /* might wait for B */ + /* start to take into account event B's context */ + /* 3 */ + + folio_unlock B + /* event B that's been valid since 3 */ + + mutex_unlock A + /* event A that's been valid since 1 */ + +There are two events. For event B, folio_unlock B, since there are no +waits between 3 and the event, event B does not create dependency. For +event A, there is a wait, folio_lock B, between 1 and the event. Which +means event A cannot be triggered if event B does not wake up the wait. +Therefore, we can say event A depends on event B, say, 'A -> B'. The +graph will look like after adding the dependency: + + A -> B + + where 'A -> B' means that event A depends on event B. + +Lastly, context Z: + + context Z + + mutex_lock A + /* might wait for A */ + /* start to take into account event A's context */ + /* 4 */ + + folio_unlock B + /* event B that's been valid since 2 */ + + mutex_unlock A + /* event A that's been valid since 4 */ + +There are also two events. For event B, folio_unlock B, there is a +wait, mutex_lock A, between 2 and the event - remind 2 is at a very +start and before the wait in timeline. Which means event B cannot be +triggered if event A does not wake up the wait. Therefore, we can say +event B depends on event A, say, 'B -> A'. The graph will look like +after adding the dependency: + + -> A -> B - + / \ + \ / + ----------- + + where 'A -> B' means that event A depends on event B. + +A new loop has been created. So DEPT can report it as a deadlock. For +event A, mutex_unlock A, since there are no waits between 4 and the +event, event A does not create dependency. That's it. + +CONCLUSION + +DEPT works well with any general synchronization mechanisms by focusing +on wait, event and its context. + + +Interpret DEPT report +--------------------- + +The following is the example in the section 'How DEPT works'. + + context X context Y context Z + + mutex_lock A + /* might wait for A */ + /* start to take into account event A's context */ + /* 1 */ + folio_lock B + /* might wait for B */ + /* start to take into account event B's context */ + /* 2 */ + + folio_lock B + /* might wait for B */ <- DEADLOCK + /* start to take into account event B's context */ + /* 3 */ + + mutex_lock A + /* might wait for A */ <- DEADLOCK + /* start to take into account + event A's context */ + /* 4 */ + + folio_unlock B + /* event B that's been valid since 2 */ + folio_unlock B + /* event B that's been valid since 3 */ + + mutex_unlock A + /* event A that's been valid since 1 */ + + mutex_unlock A + /* event A that's been valid since 4 */ + +We can Simplify this by replacing each waiting point with [W], each +point where its event's context starts with [S] and each event with [E]. +This example will look like after the replacement: + + context X context Y context Z + + [W][S] mutex_lock A + [W][S] folio_lock B + [W][S] folio_lock B <- DEADLOCK + + [W][S] mutex_lock A <- DEADLOCK + [E] folio_unlock B + [E] folio_unlock B + [E] mutex_unlock A + [E] mutex_unlock A + +DEPT uses the symbols [W], [S] and [E] in its report as described above. +The following is an example reported by DEPT for a real problem. + + Link: https://lore.kernel.org/lkml/6383cde5-cf4b-facf-6e07-1378a485657d= @I-love.SAKURA.ne.jp/#t + Link: https://lore.kernel.org/lkml/1674268856-31807-1-git-send-email-by= ungchul.park@lge.com/ + + =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D + DEPT: Circular dependency has been detected. + 6.2.0-rc1-00025-gb0c20ebf51ac-dirty #28 Not tainted + --------------------------------------------------- + summary + --------------------------------------------------- + *** DEADLOCK *** + + context A + [S] lock(&ni->ni_lock:0) + [W] folio_wait_bit_common(PG_locked_map:0) + [E] unlock(&ni->ni_lock:0) + + context B + [S] (unknown)(PG_locked_map:0) + [W] lock(&ni->ni_lock:0) + [E] folio_unlock(PG_locked_map:0) + + [S]: start of the event context + [W]: the wait blocked + [E]: the event not reachable + --------------------------------------------------- + context A's detail + --------------------------------------------------- + context A + [S] lock(&ni->ni_lock:0) + [W] folio_wait_bit_common(PG_locked_map:0) + [E] unlock(&ni->ni_lock:0) + + [S] lock(&ni->ni_lock:0): + [] ntfs3_setattr+0x54b/0xd40 + stacktrace: + ntfs3_setattr+0x54b/0xd40 + notify_change+0xcb3/0x1430 + do_truncate+0x149/0x210 + path_openat+0x21a3/0x2a90 + do_filp_open+0x1ba/0x410 + do_sys_openat2+0x16d/0x4e0 + __x64_sys_creat+0xcd/0x120 + do_syscall_64+0x41/0xc0 + entry_SYSCALL_64_after_hwframe+0x63/0xcd + + [W] folio_wait_bit_common(PG_locked_map:0): + [] truncate_inode_pages_range+0x9b0/0xf20 + stacktrace: + folio_wait_bit_common+0x5e0/0xaf0 + truncate_inode_pages_range+0x9b0/0xf20 + truncate_pagecache+0x67/0x90 + ntfs3_setattr+0x55a/0xd40 + notify_change+0xcb3/0x1430 + do_truncate+0x149/0x210 + path_openat+0x21a3/0x2a90 + do_filp_open+0x1ba/0x410 + do_sys_openat2+0x16d/0x4e0 + __x64_sys_creat+0xcd/0x120 + do_syscall_64+0x41/0xc0 + entry_SYSCALL_64_after_hwframe+0x63/0xcd + + [E] unlock(&ni->ni_lock:0): + (N/A) + --------------------------------------------------- + context B's detail + --------------------------------------------------- + context B + [S] (unknown)(PG_locked_map:0) + [W] lock(&ni->ni_lock:0) + [E] folio_unlock(PG_locked_map:0) + + [S] (unknown)(PG_locked_map:0): + (N/A) + + [W] lock(&ni->ni_lock:0): + [] attr_data_get_block+0x32c/0x19f0 + stacktrace: + attr_data_get_block+0x32c/0x19f0 + ntfs_get_block_vbo+0x264/0x1330 + __block_write_begin_int+0x3bd/0x14b0 + block_write_begin+0xb9/0x4d0 + ntfs_write_begin+0x27e/0x480 + generic_perform_write+0x256/0x570 + __generic_file_write_iter+0x2ae/0x500 + ntfs_file_write_iter+0x66d/0x1d70 + do_iter_readv_writev+0x20b/0x3c0 + do_iter_write+0x188/0x710 + vfs_iter_write+0x74/0xa0 + iter_file_splice_write+0x745/0xc90 + direct_splice_actor+0x114/0x180 + splice_direct_to_actor+0x33b/0x8b0 + do_splice_direct+0x1b7/0x280 + do_sendfile+0xb49/0x1310 + + [E] folio_unlock(PG_locked_map:0): + [] generic_write_end+0xf2/0x440 + stacktrace: + generic_write_end+0xf2/0x440 + ntfs_write_end+0x42e/0x980 + generic_perform_write+0x316/0x570 + __generic_file_write_iter+0x2ae/0x500 + ntfs_file_write_iter+0x66d/0x1d70 + do_iter_readv_writev+0x20b/0x3c0 + do_iter_write+0x188/0x710 + vfs_iter_write+0x74/0xa0 + iter_file_splice_write+0x745/0xc90 + direct_splice_actor+0x114/0x180 + splice_direct_to_actor+0x33b/0x8b0 + do_splice_direct+0x1b7/0x280 + do_sendfile+0xb49/0x1310 + __x64_sys_sendfile64+0x1d0/0x210 + do_syscall_64+0x41/0xc0 + entry_SYSCALL_64_after_hwframe+0x63/0xcd + --------------------------------------------------- + information that might be helpful + --------------------------------------------------- + CPU: 1 PID: 8060 Comm: a.out Not tainted + 6.2.0-rc1-00025-gb0c20ebf51ac-dirty #28 + Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), + BIOS Bochs 01/01/2011 + Call Trace: + + dump_stack_lvl+0xf2/0x169 + print_circle.cold+0xca4/0xd28 + ? lookup_dep+0x240/0x240 + ? extend_queue+0x223/0x300 + cb_check_dl+0x1e7/0x260 + bfs+0x27b/0x610 + ? print_circle+0x240/0x240 + ? llist_add_batch+0x180/0x180 + ? extend_queue_rev+0x300/0x300 + ? __add_dep+0x60f/0x810 + add_dep+0x221/0x5b0 + ? __add_idep+0x310/0x310 + ? add_iecxt+0x1bc/0xa60 + ? add_iecxt+0x1bc/0xa60 + ? add_iecxt+0x1bc/0xa60 + ? add_iecxt+0x1bc/0xa60 + __dept_wait+0x600/0x1490 + ? add_iecxt+0x1bc/0xa60 + ? truncate_inode_pages_range+0x9b0/0xf20 + ? check_new_class+0x790/0x790 + ? dept_enirq_transition+0x519/0x9c0 + dept_wait+0x159/0x3b0 + ? truncate_inode_pages_range+0x9b0/0xf20 + folio_wait_bit_common+0x5e0/0xaf0 + ? filemap_get_folios_contig+0xa30/0xa30 + ? dept_enirq_transition+0x519/0x9c0 + ? lock_is_held_type+0x10e/0x160 + ? lock_is_held_type+0x11e/0x160 + truncate_inode_pages_range+0x9b0/0xf20 + ? truncate_inode_partial_folio+0xba0/0xba0 + ? setattr_prepare+0x142/0xc40 + truncate_pagecache+0x67/0x90 + ntfs3_setattr+0x55a/0xd40 + ? ktime_get_coarse_real_ts64+0x1e5/0x2f0 + ? ntfs_extend+0x5c0/0x5c0 + ? mode_strip_sgid+0x210/0x210 + ? ntfs_extend+0x5c0/0x5c0 + notify_change+0xcb3/0x1430 + ? do_truncate+0x149/0x210 + do_truncate+0x149/0x210 + ? file_open_root+0x430/0x430 + ? process_measurement+0x18c0/0x18c0 + ? ntfs_file_release+0x230/0x230 + path_openat+0x21a3/0x2a90 + ? path_lookupat+0x840/0x840 + ? dept_enirq_transition+0x519/0x9c0 + ? lock_is_held_type+0x10e/0x160 + do_filp_open+0x1ba/0x410 + ? may_open_dev+0xf0/0xf0 + ? find_held_lock+0x2d/0x110 + ? lock_release+0x43c/0x830 + ? dept_ecxt_exit+0x31a/0x590 + ? _raw_spin_unlock+0x3b/0x50 + ? alloc_fd+0x2de/0x6e0 + do_sys_openat2+0x16d/0x4e0 + ? __ia32_sys_get_robust_list+0x3b0/0x3b0 + ? build_open_flags+0x6f0/0x6f0 + ? dept_enirq_transition+0x519/0x9c0 + ? dept_enirq_transition+0x519/0x9c0 + ? lock_is_held_type+0x4e/0x160 + ? lock_is_held_type+0x4e/0x160 + __x64_sys_creat+0xcd/0x120 + ? __x64_compat_sys_openat+0x1f0/0x1f0 + do_syscall_64+0x41/0xc0 + entry_SYSCALL_64_after_hwframe+0x63/0xcd + RIP: 0033:0x7f8b9e4e4469 + Code: 00 f3 c3 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 40 00 48 89 f8 48 + 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> + 3d 01 f0 ff ff 73 01 c3 48 8b 0d ff 49 2b 00 f7 d8 64 89 01 48 + RSP: 002b:00007f8b9eea4ef8 EFLAGS: 00000202 ORIG_RAX: 0000000000000055 + RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007f8b9e4e4469 + RDX: 0000000000737562 RSI: 0000000000000000 RDI: 0000000020000000 + RBP: 00007f8b9eea4f20 R08: 0000000000000000 R09: 0000000000000000 + R10: 0000000000000000 R11: 0000000000000202 R12: 00007fffa75511ee + R13: 00007fffa75511ef R14: 00007f8b9ee85000 R15: 0000000000000003 + + +Let's take a look at the summary that is the most important part. + + --------------------------------------------------- + summary + --------------------------------------------------- + *** DEADLOCK *** + + context A + [S] lock(&ni->ni_lock:0) + [W] folio_wait_bit_common(PG_locked_map:0) + [E] unlock(&ni->ni_lock:0) + + context B + [S] (unknown)(PG_locked_map:0) + [W] lock(&ni->ni_lock:0) + [E] folio_unlock(PG_locked_map:0) + + [S]: start of the event context + [W]: the wait blocked + [E]: the event not reachable + +The summary shows the following scenario: + + context A context B context ?(unknown) + + [S] folio_lock(&f1) + [S] lock(&ni->ni_lock:0) + [W] folio_wait_bit_common(PG_locked_map:0) + + [W] lock(&ni->ni_lock:0) + [E] folio_unlock(&f1) + + [E] unlock(&ni->ni_lock:0) + +Adding supplementary comments to describe DEPT's view in detail: + + context A context B context ?(unknown) + + [S] folio_lock(&f1) + /* start to take into account context + B heading for folio_unlock(&f1) */ + /* 1 */ + [S] lock(&ni->ni_lock:0) + /* start to take into account this context heading for + unlock(&ni->ni_lock:0) */ + /* 2 */ + + [W] folio_wait_bit_common(PG_locked_map:0) (=3D folio_lock(&f1)) + /* might wait for folio_unlock(&f1) */ + + [W] lock(&ni->ni_lock:0) + /* might wait for unlock(&ni->ni_lock:0) */ + + [E] folio_unlock(&f1) + /* event that's been valid since 1 */ + + [E] unlock(&ni->ni_lock:0) + /* event that's been valid since 2 */ + +Let's build up dependency graph with this report. Firstly, context A: + + context A + + [S] lock(&ni->ni_lock:0) + /* start to take into account this context heading for + unlock(&ni->ni_lock:0) */ + /* 2 */ + + [W] folio_wait_bit_common(PG_locked_map:0) (=3D folio_lock(&f1)) + /* might wait for folio_unlock(&f1) */ + + [E] unlock(&ni->ni_lock:0) + /* event that's been valid since 2 */ + +There is one interesting event, unlock(&ni->ni_lock:0). There is a +wait, folio_lock(&f1), between 2 and the event. Which means +unlock(&ni->ni_lock:0) is not reachable if folio_unlock(&f1) does not +wake up the wait. Therefore, we can say unlock(&ni->ni_lock:0) depends +on folio_unlock(&f1), say, 'unlock(&ni->ni_lock:0) -> folio_unlock(&f1)'. +The graph will look like after adding the dependency: + + unlock(&ni->ni_lock:0) -> folio_unlock(&f1) + + where 'A -> B' means that event A depends on event B. + +Secondly, context B: + + context B + + [W] lock(&ni->ni_lock:0) + /* might wait for unlock(&ni->ni_lock:0) */ + + [E] folio_unlock(&f1) + /* event that's been valid since 1 */ + +There is also one interesting event, folio_unlock(&f1). There is a +wait, lock(&ni->ni_lock:0), between 1 and the event - remind 1 is at a +very start and before the wait in timeline. Which means folio_unlock(&f1) +is not reachable if unlock(&ni->ni_lock:0) does not wake up the wait. +Therefore, we can say folio_unlock(&f1) depends on unlock(&ni->ni_lock:0), +say, 'folio_unlock(&f1) -> unlock(&ni->ni_lock:0)'. The graph will look +like after adding the dependency: + + -> unlock(&ni->ni_lock:0) -> folio_unlock(&f1) - + / \ + \ / + ------------------------------------------------ + + where 'A -> B' means that event A depends on event B. + +A new loop has been created. So DEPT can report it as a deadlock! Cool! + +CONCLUSION + +DEPT works awesome! diff --git a/Documentation/dependency/dept_api.txt b/Documentation/dependen= cy/dept_api.txt new file mode 100644 index 000000000000..8e0d5a118a46 --- /dev/null +++ b/Documentation/dependency/dept_api.txt @@ -0,0 +1,117 @@ +DEPT(DEPendency Tracker) APIs +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D + +Started by Byungchul Park + +SDT(Single-event Dependency Tracker) APIs +----------------------------------------- +Use these APIs to annotate on either wait or event. These have been +already applied into the existing synchronization primitives e.g. +waitqueue, swait, wait_for_completion(), dma fence and so on. The basic +APIs of SDT are: + + /* + * After defining 'struct dept_map map', initialize the instance. + */ + sdt_map_init(map); + + /* + * Place just before the interesting wait. + */ + sdt_wait(map); + + /* + * Place just before the interesting event. + */ + sdt_event(map); + +The advanced APIs of SDT are: + + /* + * After defining 'struct dept_map map', initialize the instance + * using an external key. + */ + sdt_map_init_key(map, key); + + /* + * Place just before the interesting timeout wait. + */ + sdt_wait_timeout(map, time); + + /* + * Use sdt_might_sleep_start() and sdt_might_sleep_end() in pair. + * Place at the start of the interesting section that might enter + * schedule() or its family that needs to be woken up by + * try_to_wake_up(). + */ + sdt_might_sleep_start(map); + + /* + * Use sdt_might_sleep_start_timeout() and sdt_might_sleep_end() in + * pair. Place at the start of the interesting section that might + * enter schedule_timeout() or its family that needs to be woken up + * by try_to_wake_up(). + */ + sdt_might_sleep_start_timeout(map, time); + + /* + * Use sdt_might_sleep_start() and sdt_might_sleep_end() in pair. + * Place at the end of the interesting section that might enter + * schedule(), schedule_timeout() or its family that needs to be + * woken up by try_to_wake_up(). + */ + sdt_might_sleep_end(); + + /* + * Use sdt_ecxt_enter() and sdt_ecxt_exit() in pair. Place at the + * start of the interesting section where the interesting event might + * be triggered. + */ + sdt_ecxt_enter(map); + + /* + * Use sdt_ecxt_enter() and sdt_ecxt_exit() in pair. Place at the + * end of the interesting section where the interesting event might + * be triggered. + */ + sdt_ecxt_exit(map); + + +LDT(Lock Dependency Tracker) APIs +--------------------------------- +Do not use these APIs directly. These are the wrappers for typical +locks, that have been already applied into major locks internally e.g. +spin lock, mutex, rwlock and so on. The APIs of LDT are: + + ldt_init(map, key, sub, name); + ldt_lock(map, sub_local, try, nest, ip); + ldt_rlock(map, sub_local, try, nest, ip, queued); + ldt_wlock(map, sub_local, try, nest, ip); + ldt_unlock(map, ip); + ldt_downgrade(map, ip); + ldt_set_class(map, name, key, sub_local, ip); + + +Raw APIs +-------- +Do not use these APIs directly. The raw APIs of dept are: + + dept_free_range(start, size); + dept_map_init(map, key, sub, name); + dept_map_reinit(map, key, sub, name); + dept_ext_wgen_init(ext_wgen); + dept_map_copy(map_to, map_from); + dept_wait(map, wait_flags, ip, wait_func, sub_local, time); + dept_stage_wait(map, key, ip, wait_func, time); + dept_request_event_wait_commit(); + dept_clean_stage(); + dept_stage_event(task, ip); + dept_ecxt_enter(map, evt_flags, ip, ecxt_func, evt_func, sub_local); + dept_ecxt_holding(map, evt_flags); + dept_request_event(map, ext_wgen); + dept_event(map, evt_flags, ip, evt_func, ext_wgen); + dept_ecxt_exit(map, evt_flags, ip); + dept_ecxt_enter_nokeep(map); + dept_key_init(key); + dept_key_destroy(key); + dept_map_ecxt_modify(map, cur_evt_flags, key, evt_flags, ip, ecxt_func,= evt_func, sub_local); --=20 2.17.1 From nobody Fri Dec 19 15:51:27 2025 Received: from invmail4.hynix.com (exvmail4.skhynix.com [166.125.252.92]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 8150B27A934; Mon, 19 May 2025 09:18:59 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=166.125.252.92 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747646343; cv=none; b=JAlQ3pkTsWR73ZZgs7QxxZoD6vXGmAXr6ODS1qrcQAmFHWPUDNtBtjvBEqTndq+VyEttOjw8r1Ebj+WjAMa69rMfy5V5tcuhck++pWY8arM9V64O2NiK+1jPgLY90bKdbtGbEPQmCZR189YSgV/VjrgtjBfWHXSk4l3oiooz8aQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747646343; c=relaxed/simple; bh=1IZJyJwmvBeB2aOIzTtB/M01VK39C8cxFBwYkS4jNho=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References; b=OUSklV3RsT1Io/wY+zyT40iQwlBiXAsJZToBcYxybpFZJxley8FqZWx9c0HB5xU0NjWFsqIY+KZApSU0kHj/qjF1CdFMGW7qrN8L6UDFOyklDJ9xzXl2Itlg847eNFvWF9ZN1+zDqsl8BPrn4A214bhBuVMQP57kUa0Z2igNBKM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com; spf=pass smtp.mailfrom=sk.com; arc=none smtp.client-ip=166.125.252.92 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=sk.com X-AuditID: a67dfc5b-681ff7000002311f-9b-682af76f4a34 From: Byungchul Park To: linux-kernel@vger.kernel.org Cc: kernel_team@skhynix.com, torvalds@linux-foundation.org, damien.lemoal@opensource.wdc.com, linux-ide@vger.kernel.org, adilger.kernel@dilger.ca, linux-ext4@vger.kernel.org, mingo@redhat.com, peterz@infradead.org, will@kernel.org, tglx@linutronix.de, rostedt@goodmis.org, joel@joelfernandes.org, sashal@kernel.org, daniel.vetter@ffwll.ch, duyuyang@gmail.com, johannes.berg@intel.com, tj@kernel.org, tytso@mit.edu, willy@infradead.org, david@fromorbit.com, amir73il@gmail.com, gregkh@linuxfoundation.org, kernel-team@lge.com, linux-mm@kvack.org, akpm@linux-foundation.org, mhocko@kernel.org, minchan@kernel.org, hannes@cmpxchg.org, vdavydov.dev@gmail.com, sj@kernel.org, jglisse@redhat.com, dennis@kernel.org, cl@linux.com, penberg@kernel.org, rientjes@google.com, vbabka@suse.cz, ngupta@vflare.org, linux-block@vger.kernel.org, josef@toxicpanda.com, linux-fsdevel@vger.kernel.org, jack@suse.cz, jlayton@kernel.org, dan.j.williams@intel.com, hch@infradead.org, djwong@kernel.org, dri-devel@lists.freedesktop.org, rodrigosiqueiramelo@gmail.com, melissa.srw@gmail.com, hamohammed.sa@gmail.com, harry.yoo@oracle.com, chris.p.wilson@intel.com, gwan-gyeong.mun@intel.com, max.byungchul.park@gmail.com, boqun.feng@gmail.com, longman@redhat.com, yskelg@gmail.com, yunseong.kim@ericsson.com, yeoreum.yun@arm.com, netdev@vger.kernel.org, matthew.brost@intel.com, her0gyugyu@gmail.com Subject: [PATCH v16 29/42] cpu/hotplug: use a weaker annotation in AP thread Date: Mon, 19 May 2025 18:18:13 +0900 Message-Id: <20250519091826.19752-30-byungchul@sk.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20250519091826.19752-1-byungchul@sk.com> References: <20250519091826.19752-1-byungchul@sk.com> X-Brightmail-Tracker: H4sIAAAAAAAAAzWSbUxTdxTG/d97e+9tZ+WmEnfBJcxGs8niCwaWs2SaJVv0P+MLiX4w24d5 I3e0ASopiuBc0k5qnJWCOmAqYgFzraWM2mIUtYpFKgVFkIrAeJeoBJCl2CqC2wqbX05+eZ7z POfLYUmVTxbLanX7RL1OSFfTCkoxsbB8VebreM1aS10UhF4dpaC0xkFD2x9VCBy1RgJGGzfB k/A4gpkHD0koKWpDUD7UR0Ktrx+Bx/YLDR0jiyAQmqTBX2Sm4XBlDQ3tY7ME9BafJKDKtRUG pGcUtBRWEFAySsPZksNEZLwgYFqyMyAZVsCw7QwDs0MJ4O/vlIGn5zM4XdZLw02PnwLftWEC Oq6X0tDv+EcGLb4mCsKWpdB2Il8G1S8raBgLSyRIoUkGHtVbCfBZl4AzL1J4ZOpvGdzLryfg yIXLBAS6byC4dXSQAJejk4aG0DgBblcRCW8vNiIYtkwwYDo+zcBZowWB2VRMQV5vEsy8iVw+ 9yoBjOedFFS/60RfrceOMgfCDeOTJM5zH8BvQ49p7AlbKdxcweO6M30MzrvVw2Craz922+Jx 5c1RApcHQzLssv9KY1fwJIOPTQQI/LK1lUn+6DvFlyliujZb1K/ZsFuhaZRmiMxBec7Vx8+R Ac0yx5Cc5blEvmPohew9m7yD5BzT3Cd8V9f0PEdzH/Pu/GeRHQVLcp0f8E/OdaM5YzG3hbf2 jkeYYSluBX9JnFOV3Od8U3uQ/K8yjq9y1s+zPKL3mBvmkyouiQ9UlVFzlTxXIOdN1ub/AzH8 HVsXVYiUVrTAjlRaXXaGoE1PXK3J1WlzVu/Zm+FCke+Sfp79/hoKtu3wIo5F6oVKp2elRiUT srNyM7yIZ0l1tNLu/lSjUqYIuQdF/d4f9PvTxSwvWspS6g+V68IHUlRcqrBPTBPFTFH/3iVY eawBJU+pDDP6UvPtXbVLTOYvjN2PXuM0erIvTjpI7woL7Tk/qakbzd8w73TQxZ2y7Kxbdvlr qUMw3v9xc3TN+TWnUpNsfqEgcdtzIaaw9vi9EUPrmHfr3W+nY9JUUByVWk76t/8e5zz0W+zG TX+uH1hufxoYGOoLXvlre1KooGWzKUpNZWmEhHhSnyX8C0ns3bFZAwAA X-Brightmail-Tracker: H4sIAAAAAAAAAzWSa0hTcRjG+5+708Vxih2MLiykMrpIGS8UEfSh05WiILKgRp7acNPYyrQI XJtdvCwVpmlp89Lyskq3KEsnNnG58lbqLFMzi0oyJXWmadY0+vLw43l4n+fLy+CSYjKYUUSf EdTRMqWUEhGivZt0q2N+hsrXdRQBeMauEnDrgYWC1vtlCCwPtRgM1G+HzvFBBFNNLThkGVsR 5H/oweGhsxeBvfgSBW2f5kO7Z5gClzGZAl3hAwpefZvGoDszA4My6x54b/5MwMu0AgyyBii4 maXDvPIVg0lzKQ3mhBDoL86hYfpDGLh63STU5bpIsHetguy8bgqq7S4CnJX9GLQ9vUVBr+UP CS+dDQSMGxZCa3oqCfeGCij4Nm7GwewZpuF1rQkDpykIyvXe1sujMyQ8T63F4HJRBQbtb6sQ 1Fztw8BqcVNQ5xnEwGY14vDrbj2CfsN3GhJTJmm4qTUgSE7MJEDfHQ5TE97l3LEw0N4uJ+De bzfauoW35FkQXzc4jPN62zn+l6eD4u3jJoJ/UcDxT3J6aF5f00XzJutZ3lYcyhdWD2B8/oiH 5K2l1yjeOpJB80nf2zF+qLmZ3rcoQrQ5UlAqYgX12i3HRfJ68xR2us8n7nHHF5SApukk5MNw 7AYu0dGHzzLFLufevJmc40B2KWdL/UwmIRGDs25frjP3LZoNAtjdnKl70Ms0Q7AhXIkw64rZ jVzDqxH8X+USrqy8do59vH5Xct3cpYQN59rL8og0JDKheaUoUBEdq5IplOFrNFHy+GhF3JoT MSor8v6P+eJ0eiUaa9vuQCyDpH7icvtKuYSUxWriVQ7EMbg0UFxqWyGXiCNl8ecFdcwx9Vml oHGghQwhXSDeeUg4LmFPyc4IUYJwWlD/TzHGJzgBqVK2fV18p2vGKQ3I8WsJGCnRaInrKzPk lVSUr2iXYvjR0T+uknVKldKQfUTz8Yfbf2jZ2LvmC5eCnlFBui+10vzRkpYJ9Y2qLMeVqk79 jvXVh43aif0/Uq0VGy2GiPem4Be+nSESnbFR4y9qOnJRHXDS/2PFqF7f0Fg4EXXA76CU0Mhl YaG4WiP7C409OyQ7AwAA X-CFilter-Loop: Reflected Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" cb92173d1f0 ("locking/lockdep, cpu/hotplug: Annotate AP thread") was introduced to make lockdep_assert_cpus_held() work in AP thread. However, the annotation is too strong for that purpose. We don't have to use more than try lock annotation for that. rwsem_acquire() implies: 1. might be a waiter on contention of the lock. 2. enter to the critical section of the lock. All we need in here is to act 2, not 1. So trylock version of annotation is sufficient for that purpose. Now that dept partially relies on lockdep annotaions, dept interpets rwsem_acquire() as a potential wait and might report a deadlock by the wait. Replace it with trylock version of annotation. Signed-off-by: Byungchul Park --- kernel/cpu.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/kernel/cpu.c b/kernel/cpu.c index b08bb34b1718..3eecd0a7b5b3 100644 --- a/kernel/cpu.c +++ b/kernel/cpu.c @@ -537,7 +537,7 @@ int lockdep_is_cpus_held(void) =20 static void lockdep_acquire_cpus_lock(void) { - rwsem_acquire(&cpu_hotplug_lock.dep_map, 0, 0, _THIS_IP_); + rwsem_acquire(&cpu_hotplug_lock.dep_map, 0, 1, _THIS_IP_); } =20 static void lockdep_release_cpus_lock(void) --=20 2.17.1 From nobody Fri Dec 19 15:51:27 2025 Received: from invmail4.hynix.com (exvmail4.hynix.com [166.125.252.92]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 0A60B26D4D4; Mon, 19 May 2025 09:18:59 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=166.125.252.92 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747646344; cv=none; b=cLGiDsr1U5QDhPatgDKiAAjDh5OGAJRwuRFDuAYN4IneQukEROF8Syl82gq4/TWJGkWCmypuxD++dvTYFHnJ4/ZGlmbjGNnVLbTZj7ORl8bixHcR+J6DRVXzOogTWRFqF6fha08OpwScA15FhwearMBLxBDLam8wcVP8kXSFPf4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747646344; c=relaxed/simple; bh=WLFy2XrzC3m864Tg26cLbN8q8gwXnUddJfJFmRKv6UE=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References; b=b4SXLmtA3OZuPuyRv2d0DiMyMDORBEaViTe/np+pNMT8hkWFp+MArL6X8kG2JIN/iRNE6oae5hW8Ysvxt0k2sfeUa+STrk6+vQBqSlbK3pQj9SKy8DXem4o6h2vo8HJkVj+bJVcVvGqtUj8LWKGO4yD1EDZeInadwHBYpsARn80= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com; spf=pass smtp.mailfrom=sk.com; arc=none smtp.client-ip=166.125.252.92 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=sk.com X-AuditID: a67dfc5b-681ff7000002311f-aa-682af7708138 From: Byungchul Park To: linux-kernel@vger.kernel.org Cc: kernel_team@skhynix.com, torvalds@linux-foundation.org, damien.lemoal@opensource.wdc.com, linux-ide@vger.kernel.org, adilger.kernel@dilger.ca, linux-ext4@vger.kernel.org, mingo@redhat.com, peterz@infradead.org, will@kernel.org, tglx@linutronix.de, rostedt@goodmis.org, joel@joelfernandes.org, sashal@kernel.org, daniel.vetter@ffwll.ch, duyuyang@gmail.com, johannes.berg@intel.com, tj@kernel.org, tytso@mit.edu, willy@infradead.org, david@fromorbit.com, amir73il@gmail.com, gregkh@linuxfoundation.org, kernel-team@lge.com, linux-mm@kvack.org, akpm@linux-foundation.org, mhocko@kernel.org, minchan@kernel.org, hannes@cmpxchg.org, vdavydov.dev@gmail.com, sj@kernel.org, jglisse@redhat.com, dennis@kernel.org, cl@linux.com, penberg@kernel.org, rientjes@google.com, vbabka@suse.cz, ngupta@vflare.org, linux-block@vger.kernel.org, josef@toxicpanda.com, linux-fsdevel@vger.kernel.org, jack@suse.cz, jlayton@kernel.org, dan.j.williams@intel.com, hch@infradead.org, djwong@kernel.org, dri-devel@lists.freedesktop.org, rodrigosiqueiramelo@gmail.com, melissa.srw@gmail.com, hamohammed.sa@gmail.com, harry.yoo@oracle.com, chris.p.wilson@intel.com, gwan-gyeong.mun@intel.com, max.byungchul.park@gmail.com, boqun.feng@gmail.com, longman@redhat.com, yskelg@gmail.com, yunseong.kim@ericsson.com, yeoreum.yun@arm.com, netdev@vger.kernel.org, matthew.brost@intel.com, her0gyugyu@gmail.com Subject: [PATCH v16 30/42] fs/jbd2: use a weaker annotation in journal handling Date: Mon, 19 May 2025 18:18:14 +0900 Message-Id: <20250519091826.19752-31-byungchul@sk.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20250519091826.19752-1-byungchul@sk.com> References: <20250519091826.19752-1-byungchul@sk.com> X-Brightmail-Tracker: H4sIAAAAAAAAAzWSfVDLcRzHfX+P2zR+Nw/9xOFGh5yHkPtwiCN+/zhP/8Vhpx/bqeyWUpy7 poRWCT1cD5tVTLcW2dzJw5J2pkSGWQ8q6Tqnq+kubT2ssHL++dz7Pu97vT7/fAS4pJ4MEihi z/KqWFm0lBIRIndAySrlcIh8rbl6DniGrhJQ/MBEgeN+BQLTIzUGva/2QLO3H4Hv3Xsc8nMd CEq+deDwyN6JwFp+iYJPPTPA6RmgoCFXQ0FK2QMKPvSNY9CedxODCvNe+Gr4TkBjdikG+b0U FOWnYP7xA4NRg5EGQ3IwdJcX0jD+LRQaOl0kWNtWQoGunYLn1gYC7NXdGHx6WkxBp+kPCY32 egK8WfPBcSOThMqfpRT0eQ04GDwDNHys1WNg18+FqlS/MO3XbxJeZ9ZikHbnIQbO1mcIaq52 YWA2uSiwefoxsJhzcRi79wpBd5abhssZozQUqbMQaC7nEZDaHga+Ef9l7VAoqG9XEVA54ULb t3ImnQlxtv4BnEu1nOPGPJ8pzurVE9ybUpZ7UthBc6k1bTSnN8dzlvIQrux5L8aVDHpIzmy8 RnHmwZs0l+52YtzPpiZ6/4JI0ZYoPlqRwKvWbDsukmucBZhyZHqi9noTSkbDwnQkFLDMBnbE VUD9zwa9mp7MFLOMbWkZxSfzbGYxa8n8TqYjkQBnXNPZZm0rmixmMftY291bUzDBBLOfy1Km YDGzkbV9sZP/pIvYiqraKZHQv2/T2KZYCRPGOit0xKSUZfKFbFGRG/0D5rEvy1uIbCTWo2lG JFHEJsTIFNEbVsuTYhWJq0+ciTEj/38ZLo4frkaDjkN1iBEgaYC4yrpCLiFlCXFJMXWIFeDS 2WKjZblcIo6SJZ3nVWeOqeKj+bg6NF9ASAPF67znoiTMKdlZ/jTPK3nV/xYTCIOSUfHBx2+Q 8q16FNdf2aRxBy2pFy1JrF0HkqGeI1927f7wokt7Mrsy3NjlXtU3FqG55NuREam1f81b1PM7 40ikL1gXYc8IW3hvYdmBzTkBD5tTjnb7wpUdhojSxm0XdK0TNbNOXPDFzwx0z5h4u5Nfej8t J3D9MJOnd4Rv5hdHaXN4KREnl4WG4Ko42V+wLt0OWwMAAA== X-Brightmail-Tracker: H4sIAAAAAAAAAzWSa2yLcRTG/d9rWzpvauGNu8bCtrjUjMPELcHfbUgkEiI09kYbuzQts0mw 6SysNkam1Dq1UtN1Nq3FXCrLmpW5zFgVM6MLYrFZsM42c+kmvpz8cp48v/PliEiZnR4tUifv FrTJykQ5I6Ek8XH6aZofUaqZRhOCYNcRCgrLHQw0XC1F4LieSUBb7Qp40d2O4OfjJyQYCxoQ XAi8IeG6twWBu+QQA43vw8AX7GSgrsDAgN5azsDTz/0ENJ8+SUCpcy28tX2k4OGJYgKMbQyc M+qJ0PhEQK/NzoItIwJaS0ws9AcUUNfip8FjrqPB3RQNZ4uaGbjjrqPAW9VKQOOtQgZaHH9o eOi9T0F33hhoyM+loexLMQOfu20k2IKdLDyrthDgtYyEiqyQNfv7bxru5VYTkH3xGgG+V7cR 3D3yjgCnw8+AJ9hOgMtZQELf5VoErXkdLBw+1svCucw8BIbDpynIao6Fnz2hy+YuBWSer6Cg 7JcfLV6IHUUOhD3tnSTOcu3FfcHnDHZ3Wyj8oJjHN01vWJx1t4nFFuce7CqJwtY7bQS+8C1I Y6f9KIOd306yOKfDR+Av9fXs+nGbJQsShER1qqCdsXC7RGXwnSU0PUPTzMfrUQb6Ic5BYhHP zeZtlkx2gBluCv/yZS85wOHcRN6V+5HOQRIRyfmH8i/Mr9BAMIJbx3sunWIGmOIi+OdW/WBZ ys3hPa+99D/pBL60onpQJA7tmwyewa6Mi+V9pUXUCSSxoCF2FK5OTk1SqhNjp+t2qdKT1WnT d6QkOVHog2z7+/OrUFfjihrEiZB8mLTCHamS0cpUXXpSDeJFpDxcandNVcmkCcr0fYI2ZZt2 T6Kgq0FjRJR8lHTVJmG7jNup3C3sEgSNoP2fEiLx6Aykiig0tj26X2u6MT8gObY8Hy/23ypc NLY1Ol9cvnLucSYu+rHEHplS1qxWfM0eFh/TEzN364E18yrDzqSNOxS25coG6+39ZYaYSZM0 Vcv+jC+eVcmFjVyScLFjtiF2Y8H3QKDpw2qr5qDeMGSReZZx3gTT5IRRBxUPhnsqy+P6Kocv WSqndCqlIorU6pR/AZU0VWc9AwAA X-CFilter-Loop: Reflected Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" jbd2 journal handling code doesn't want jbd2_might_wait_for_commit() to be placed between start_this_handle() and stop_this_handle(). So it marks the region with rwsem_acquire_read() and rwsem_release(). However, the annotation is too strong for that purpose. We don't have to use more than try lock annotation for that. rwsem_acquire_read() implies: 1. might be a waiter on contention of the lock. 2. enter to the critical section of the lock. All we need in here is to act 2, not 1. So trylock version of annotation is sufficient for that purpose. Now that dept partially relies on lockdep annotaions, dept interpets rwsem_acquire_read() as a potential wait and might report a deadlock by the wait. Replace it with trylock version of annotation. Signed-off-by: Byungchul Park --- fs/jbd2/transaction.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/fs/jbd2/transaction.c b/fs/jbd2/transaction.c index cbc4785462f5..7de227a2a6f8 100644 --- a/fs/jbd2/transaction.c +++ b/fs/jbd2/transaction.c @@ -441,7 +441,7 @@ static int start_this_handle(journal_t *journal, handle= _t *handle, read_unlock(&journal->j_state_lock); current->journal_info =3D handle; =20 - rwsem_acquire_read(&journal->j_trans_commit_map, 0, 0, _THIS_IP_); + rwsem_acquire_read(&journal->j_trans_commit_map, 0, 1, _THIS_IP_); jbd2_journal_free_transaction(new_transaction); /* * Ensure that no allocations done while the transaction is open are --=20 2.17.1 From nobody Fri Dec 19 15:51:27 2025 Received: from invmail4.hynix.com (exvmail4.hynix.com [166.125.252.92]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 0B13627AC37; Mon, 19 May 2025 09:18:59 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=166.125.252.92 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747646344; cv=none; b=aHlM6o4CsqVjhI2nJC/o5yRgL+IAuMBqcT1G0LUJL8m5kqSBLuK4wErNbcxYwiBg4MHOTN2nah/wpdevsxrS1yYu5UriqapDBZzsqcCXx4CnKBVDbzKht52MHEV2ecmng73N/BthE02b/nRW4JCo6Ircc5B+BTOOtLCEQaeiQQI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747646344; c=relaxed/simple; bh=YIhasWAI2DFiyLXsGvo/xd87ErP0zWnVDexTxa/J6wA=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References; b=fNKmTGygENd3ji5ff2pbd9Rr4xYYSYTZx9nYiYBxlu7Lf/m2NDgAEfaeZWy2FYyEsC16ZSBe0bqKQ+7ONCx6Urs/VDmzZOkP+7imU2936Go+6CfRVUxUjcp4XDd6HcMZ23KUnvFC0jbwhCCdOhJIHavTtl3lnyyqi4o08rNqwOQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com; spf=pass smtp.mailfrom=sk.com; arc=none smtp.client-ip=166.125.252.92 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=sk.com X-AuditID: a67dfc5b-681ff7000002311f-b9-682af770961a From: Byungchul Park To: linux-kernel@vger.kernel.org Cc: kernel_team@skhynix.com, torvalds@linux-foundation.org, damien.lemoal@opensource.wdc.com, linux-ide@vger.kernel.org, adilger.kernel@dilger.ca, linux-ext4@vger.kernel.org, mingo@redhat.com, peterz@infradead.org, will@kernel.org, tglx@linutronix.de, rostedt@goodmis.org, joel@joelfernandes.org, sashal@kernel.org, daniel.vetter@ffwll.ch, duyuyang@gmail.com, johannes.berg@intel.com, tj@kernel.org, tytso@mit.edu, willy@infradead.org, david@fromorbit.com, amir73il@gmail.com, gregkh@linuxfoundation.org, kernel-team@lge.com, linux-mm@kvack.org, akpm@linux-foundation.org, mhocko@kernel.org, minchan@kernel.org, hannes@cmpxchg.org, vdavydov.dev@gmail.com, sj@kernel.org, jglisse@redhat.com, dennis@kernel.org, cl@linux.com, penberg@kernel.org, rientjes@google.com, vbabka@suse.cz, ngupta@vflare.org, linux-block@vger.kernel.org, josef@toxicpanda.com, linux-fsdevel@vger.kernel.org, jack@suse.cz, jlayton@kernel.org, dan.j.williams@intel.com, hch@infradead.org, djwong@kernel.org, dri-devel@lists.freedesktop.org, rodrigosiqueiramelo@gmail.com, melissa.srw@gmail.com, hamohammed.sa@gmail.com, harry.yoo@oracle.com, chris.p.wilson@intel.com, gwan-gyeong.mun@intel.com, max.byungchul.park@gmail.com, boqun.feng@gmail.com, longman@redhat.com, yskelg@gmail.com, yunseong.kim@ericsson.com, yeoreum.yun@arm.com, netdev@vger.kernel.org, matthew.brost@intel.com, her0gyugyu@gmail.com Subject: [PATCH v16 31/42] dept: assign dept map to mmu notifier invalidation synchronization Date: Mon, 19 May 2025 18:18:15 +0900 Message-Id: <20250519091826.19752-32-byungchul@sk.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20250519091826.19752-1-byungchul@sk.com> References: <20250519091826.19752-1-byungchul@sk.com> X-Brightmail-Tracker: H4sIAAAAAAAAAzWSa0yTZxTHfd47lZrXjuijYGaaEDdMFYzY88EZk2l8YrLMxMREjbpuvLGN LWKLCCbLQBGVSoMiMG9YQEtD66itiZdZglQLaERQpEgQsSpZJ7dh2w1FXUH9cvLL+Z/zO1+O QCta2fmCLjNbMmZq9EpOxsiG46tVWf+maFPf2NQQCR9h4GyDk4OOPxwInFcKKAjdWQeB6BCC d/cf0FBZ3oGg+vlTGq74+xF47Qc4ePRyFnRFRjloKzdzcLC2gYPO15MU9FWcoMDh/gGe2QYZ uFdaQ0FliIMzlQepWPmLgglbPQ+2/GQI2k/zMPk8Ddr6u1nw9i6GU1V9HNz0tjHgvxak4NGN sxz0Oz+ycM/fykDUkggdx0tYuDRSw8HrqI0GW2SUh4dNVgr81jngKowJi958YKGlpImCoguX Keh68ieCxiMDFLid3Rz4IkMUeNzlNLytu4MgaBnm4dCxCR7OFFgQmA9VMFDYlw7v/otdPhdO g4LzLgYuve9Gq78jzionIr6hUZoUevaRt5HHHPFGrQy5W4PJ9dNPeVLY2MsTq3sv8dhTSO3N EEWqxyMscdcf5Yh7/ARPioe7KDLS3s5vSNoiW5kh6XU5knHpqp9k2tqBMZQVUOeOTf7N5qN+ VTGKE7C4HD8Mt1Bf+J+TL6aZExfhnp4JeooTxIXYUzLIFiOZQIvdM3Hg3BM0FXwlbsPvexqn mRGT8cvyID/FcnEFtlSHPku/xg5X07QoLtbvNfum5xViOu5yVDFTUiyWxWHnhIX7tDAP37L3 MKVIbkUz6pFCl5lj0Oj0y5do8zJ1uUt+2W1wo9h/2X6d3HoNjXdsbEaigJTxcpf3W62C1eSY 8gzNCAu0MkFe7/lGq5BnaPL2S8bdO4x79ZKpGSUKjHKufFl0X4ZC3KnJlnZJUpZk/JJSQtz8 fJQtpeJjnSrf7c7AesgvYneWPUZXRxrC5kFYIWTnOlLjfbWdJ0/pAy0H7qevPRosTUxX5+GK 7c+2JiSNLuM572GVuXl/HTNzU3z7QOXGV78ZdgUO3yrIKFk1Vqa/vfmj4XuiDq3ZHjLZk3+W 1y1QXdyj/j3UmlzlZ11JYfTjbCVj0mrSUmijSfM/8QQ2DVsDAAA= X-Brightmail-Tracker: H4sIAAAAAAAAAzWSfUzMcRzHfb+/p+s4fjuN34qxMyRT2hwfY3nY6Dvk4R+MmW75zZ0ed3dS zFZKo3SLqTyVq+y0u4vrzoa4pDiOSZRKUoqZdOShi3Lhyvzz2Wuf92evzz9vCSU3M0ESTaJe 1Caq4hWslJZuXJa5IPlHqHqhpzEAvAPHaLhw1cpC4xULAuu1DAy996OgddCD4NeTpxQUFTQi KO1+TcE1VycCZ8URFpreTYRmbz8L7oJcFjLLr7LwrM+HoaPwFAaLPRq6TO9peJxfhqGol4Xz RZnYPz5gGDKZOTClz4aeinMc+LojwN3ZwkB9sZsBZ/t8OFvSwcJtp5sG140eDE3VF1jotP5h 4LHrIQ2DhmBoPJnHQOXnMhb6Bk0UmLz9HDyvNWJwGaeALctvzf7+m4EHebUYsi9VYWh+eQtB zbE3GOzWFhbqvR4MDnsBBcOX7yPoMXzi4OiJIQ7OZxgQ5B4tpCGrQwm/fvo/Fw9EQMZFGw2V Iy1oZSSxllgRqff0UyTLcYAMe1+wxDlopMmjMoHcPPeaI1k17Rwx2vcTR0UoKb/di0npNy9D 7ObjLLF/O8WRnE/NmHxuaOA2T98hXb5HjNekiNrwyBipuvzNF5TcuiT1i+8jk446F+SgAInA LxK+nn6LR5nl5wptbUPUKAfyMwVH3nsmB0klFN8yXmgtfolGg8n8LmGkrWaMaX628K6ghxtl Gb9YMJT24n/SGYLFVjsmCvDv23Prx+7lvFJotpTQ+UhqROPMKFCTmJKg0sQrw3Rx6rRETWpY bFKCHfkbZDrsO3kDDTRF1SFeghQTZDbnPLWcUaXo0hLqkCChFIEysyNELZftUaUdFLVJu7X7 40VdHQqW0IqpsnXbxBg5v1elF+NEMVnU/k+xJCAoHc1Syyb6fDGXZbp74UsMO+92BYfFXp+q Lt8effbtNoIyHu17vrrK69Gv6ftTTc95kW+SrjqyN3B9Q/vWV6UrTgwob0Zmb+Ydu1zdXfq4 YVg61xM+LarK/SCkmlg3rtWzkcIhRYVpfVDXmexN0TXKSVs2hMpi19g+WHTW8KTCOyOpClqn VkWEUlqd6i9J66ZnPQMAAA== X-CFilter-Loop: Reflected Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Resolved the following false positive by introducing explicit dept map and annotations for dealing with this case: *** DEADLOCK *** context A [S] (unknown)(:0) [W] lock(&mm->mmap_lock:0) [E] try_to_wake_up(:0) context B [S] lock(&mm->mmap_lock:0) [W] mmu_interval_read_begin(:0) [E] unlock(&mm->mmap_lock:0) [S]: start of the event context [W]: the wait blocked [E]: the event not reachable dept already tracks dependencies between scheduler sleep and ttwu based on internal timestamp called wgen. However, in case that more than one event contexts are overwrapped, dept has chance to wrongly guess the start of the event context like the following: context A: lock L context A: mmu_notifier_invalidate_range_start() context B: lock L' context B: mmu_interval_read_begin() : wait <- here is the start of the event context of C. context B: unlock L' context C: lock L'' context C: mmu_notifier_invalidate_range_start() context A: mmu_notifier_invalidate_range_end() context A: unlock L context C: mmu_notifier_invalidate_range_end() : ttwu <- here is the end of the event context of C. dept observes a wait, lock L'' within the event context of C. Which causes a false positive dept report. context C: unlock L'' By explicitly annotating the interesting event context range, make dept work with more precise information like: context A: lock L context A: mmu_notifier_invalidate_range_start() context B: lock L' context B: mmu_interval_read_begin() : wait context B: unlock L' context C: lock L'' context C: mmu_notifier_invalidate_range_start() <- here is the start of the event context of C. context A: mmu_notifier_invalidate_range_end() context A: unlock L context C: mmu_notifier_invalidate_range_end() : ttwu <- here is the end of the event context of C. dept doesn't observe the wait, lock L'' within the event context of C. context C is responsible only for the range delimited by mmu_notifier_invalidate_range_{start,end}(). context C: unlock L'' Signed-off-by: Byungchul Park --- include/linux/mmu_notifier.h | 26 ++++++++++++++++++++++++++ mm/mmu_notifier.c | 31 +++++++++++++++++++++++++++++-- 2 files changed, 55 insertions(+), 2 deletions(-) diff --git a/include/linux/mmu_notifier.h b/include/linux/mmu_notifier.h index bc2402a45741..1e256f5305b7 100644 --- a/include/linux/mmu_notifier.h +++ b/include/linux/mmu_notifier.h @@ -428,6 +428,14 @@ static inline int mmu_notifier_test_young(struct mm_st= ruct *mm, return 0; } =20 +#ifdef CONFIG_DEPT +void mmu_notifier_invalidate_dept_ecxt_start(struct mmu_notifier_range *ra= nge); +void mmu_notifier_invalidate_dept_ecxt_end(struct mmu_notifier_range *rang= e); +#else +static inline void mmu_notifier_invalidate_dept_ecxt_start(struct mmu_noti= fier_range *range) {} +static inline void mmu_notifier_invalidate_dept_ecxt_end(struct mmu_notifi= er_range *range) {} +#endif + static inline void mmu_notifier_invalidate_range_start(struct mmu_notifier_range *range) { @@ -439,6 +447,12 @@ mmu_notifier_invalidate_range_start(struct mmu_notifie= r_range *range) __mmu_notifier_invalidate_range_start(range); } lock_map_release(&__mmu_notifier_invalidate_range_start_map); + + /* + * From now on, waiters could be there by this start until + * mmu_notifier_invalidate_range_end(). + */ + mmu_notifier_invalidate_dept_ecxt_start(range); } =20 /* @@ -459,6 +473,12 @@ mmu_notifier_invalidate_range_start_nonblock(struct mm= u_notifier_range *range) ret =3D __mmu_notifier_invalidate_range_start(range); } lock_map_release(&__mmu_notifier_invalidate_range_start_map); + + /* + * From now on, waiters could be there by this start until + * mmu_notifier_invalidate_range_end(). + */ + mmu_notifier_invalidate_dept_ecxt_start(range); return ret; } =20 @@ -470,6 +490,12 @@ mmu_notifier_invalidate_range_end(struct mmu_notifier_= range *range) =20 if (mm_has_notifiers(range->mm)) __mmu_notifier_invalidate_range_end(range); + + /* + * The event context that has been started by + * mmu_notifier_invalidate_range_start() ends. + */ + mmu_notifier_invalidate_dept_ecxt_end(range); } =20 static inline void mmu_notifier_arch_invalidate_secondary_tlbs(struct mm_s= truct *mm, diff --git a/mm/mmu_notifier.c b/mm/mmu_notifier.c index fc18fe274505..850d75952f98 100644 --- a/mm/mmu_notifier.c +++ b/mm/mmu_notifier.c @@ -46,6 +46,7 @@ struct mmu_notifier_subscriptions { unsigned long active_invalidate_ranges; struct rb_root_cached itree; wait_queue_head_t wq; + struct dept_map dmap; struct hlist_head deferred_list; }; =20 @@ -165,6 +166,25 @@ static void mn_itree_inv_end(struct mmu_notifier_subsc= riptions *subscriptions) wake_up_all(&subscriptions->wq); } =20 +#ifdef CONFIG_DEPT +void mmu_notifier_invalidate_dept_ecxt_start(struct mmu_notifier_range *ra= nge) +{ + struct mmu_notifier_subscriptions *subscriptions =3D + range->mm->notifier_subscriptions; + + if (subscriptions) + sdt_ecxt_enter(&subscriptions->dmap); +} +void mmu_notifier_invalidate_dept_ecxt_end(struct mmu_notifier_range *rang= e) +{ + struct mmu_notifier_subscriptions *subscriptions =3D + range->mm->notifier_subscriptions; + + if (subscriptions) + sdt_ecxt_exit(&subscriptions->dmap); +} +#endif + /** * mmu_interval_read_begin - Begin a read side critical section against a = VA * range @@ -246,9 +266,12 @@ mmu_interval_read_begin(struct mmu_interval_notifier *= interval_sub) */ lock_map_acquire(&__mmu_notifier_invalidate_range_start_map); lock_map_release(&__mmu_notifier_invalidate_range_start_map); - if (is_invalidating) + if (is_invalidating) { + sdt_might_sleep_start(&subscriptions->dmap); wait_event(subscriptions->wq, READ_ONCE(subscriptions->invalidate_seq) !=3D seq); + sdt_might_sleep_end(); + } =20 /* * Notice that mmu_interval_read_retry() can already be true at this @@ -625,6 +648,7 @@ int __mmu_notifier_register(struct mmu_notifier *subscr= iption, =20 INIT_HLIST_HEAD(&subscriptions->list); spin_lock_init(&subscriptions->lock); + sdt_map_init(&subscriptions->dmap); subscriptions->invalidate_seq =3D 2; subscriptions->itree =3D RB_ROOT_CACHED; init_waitqueue_head(&subscriptions->wq); @@ -1070,9 +1094,12 @@ void mmu_interval_notifier_remove(struct mmu_interva= l_notifier *interval_sub) */ lock_map_acquire(&__mmu_notifier_invalidate_range_start_map); lock_map_release(&__mmu_notifier_invalidate_range_start_map); - if (seq) + if (seq) { + sdt_might_sleep_start(&subscriptions->dmap); wait_event(subscriptions->wq, mmu_interval_seq_released(subscriptions, seq)); + sdt_might_sleep_end(); + } =20 /* pairs with mmgrab in mmu_interval_notifier_insert() */ mmdrop(mm); --=20 2.17.1 From nobody Fri Dec 19 15:51:27 2025 Received: from invmail4.hynix.com (exvmail4.hynix.com [166.125.252.92]) by smtp.subspace.kernel.org (Postfix) with ESMTP id DA41827F724; Mon, 19 May 2025 09:19:00 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=166.125.252.92 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747646345; cv=none; b=vAGVVAtN6YMZ5BXV+Sk0Twae6rSo2f1frjyfZC2IpiSuDXKH86vzkdqo9r75yRLecf1dA1r0EzNNDpg85RZ25h2FlL8/LUDoJziEisFrTqnYAtaUQBU4pCGcbEfncLd0zo9euxWply9ON4fY0zn6ejZw+2FFS6+kpS7oQoBkIuo= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747646345; c=relaxed/simple; bh=4lmBLhJUCTN3YqRABT7fFsjPpJXFoz041muQQWVH95M=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References; b=BkaaF6W8L9JJdOh5kVdJUirO1aMrmSqUw0In/AWt/VQlN5GxboGGQZ4Y63tVeppLB/jMEF306BiJWtEj2y4QDu6wytGmpQSwbJ9LSvTUF1bwSqWxbFfMJk7j651Sy0002lDYGbXRsIRdHJjy7xUm4xpdy/ct7Yk1HjU7QWWHGQU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com; spf=pass smtp.mailfrom=sk.com; arc=none smtp.client-ip=166.125.252.92 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=sk.com X-AuditID: a67dfc5b-681ff7000002311f-c8-682af770481e From: Byungchul Park To: linux-kernel@vger.kernel.org Cc: kernel_team@skhynix.com, torvalds@linux-foundation.org, damien.lemoal@opensource.wdc.com, linux-ide@vger.kernel.org, adilger.kernel@dilger.ca, linux-ext4@vger.kernel.org, mingo@redhat.com, peterz@infradead.org, will@kernel.org, tglx@linutronix.de, rostedt@goodmis.org, joel@joelfernandes.org, sashal@kernel.org, daniel.vetter@ffwll.ch, duyuyang@gmail.com, johannes.berg@intel.com, tj@kernel.org, tytso@mit.edu, willy@infradead.org, david@fromorbit.com, amir73il@gmail.com, gregkh@linuxfoundation.org, kernel-team@lge.com, linux-mm@kvack.org, akpm@linux-foundation.org, mhocko@kernel.org, minchan@kernel.org, hannes@cmpxchg.org, vdavydov.dev@gmail.com, sj@kernel.org, jglisse@redhat.com, dennis@kernel.org, cl@linux.com, penberg@kernel.org, rientjes@google.com, vbabka@suse.cz, ngupta@vflare.org, linux-block@vger.kernel.org, josef@toxicpanda.com, linux-fsdevel@vger.kernel.org, jack@suse.cz, jlayton@kernel.org, dan.j.williams@intel.com, hch@infradead.org, djwong@kernel.org, dri-devel@lists.freedesktop.org, rodrigosiqueiramelo@gmail.com, melissa.srw@gmail.com, hamohammed.sa@gmail.com, harry.yoo@oracle.com, chris.p.wilson@intel.com, gwan-gyeong.mun@intel.com, max.byungchul.park@gmail.com, boqun.feng@gmail.com, longman@redhat.com, yskelg@gmail.com, yunseong.kim@ericsson.com, yeoreum.yun@arm.com, netdev@vger.kernel.org, matthew.brost@intel.com, her0gyugyu@gmail.com Subject: [PATCH v16 32/42] dept: assign unique dept_key to each distinct dma fence caller Date: Mon, 19 May 2025 18:18:16 +0900 Message-Id: <20250519091826.19752-33-byungchul@sk.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20250519091826.19752-1-byungchul@sk.com> References: <20250519091826.19752-1-byungchul@sk.com> X-Brightmail-Tracker: H4sIAAAAAAAAAzWSbUxTZxTHfZ57e++lo+amEr2+JHM1vrEMZcJyYoYhi4k3U7cZPyj6Qeu4 WaulaMvrFhMQJKzYoijiVKAUrbUUwVs0dVLCKHZDI5ZReWkQB1vcULAGbbciG2s1fjn55X/O +Z0vhyHkXskSRq3NEXRapUZBSUnpVLzloyN/J6rWz2kh9KqchIstDgp815oQONqKMUzc2QKD 4UkEr+8/IKCm2oegYewRAW3eUQRu2zEK+v+YD/5QkIKe6goKShpbKOh7Noth5GwVhiZxOzy2 PiHh3kkLhpoJCi7UlOBo+QtDxGqnwVq0EsZt52mYHUuGntEBCbgDH8IPdSMUtLt7SPC6xjH0 /3iRglHHnATueX8hIWxaCr5TRgk0P7dQ8CxsJcAaCtLwa6cZg9e8EFpLo8Kyl/9J4GdjJ4ay S9cx+IdvI+go/w2D6BigwBOaxOAUqwmYuXIHwbhpiobjJyI0XCg2Iag4fpaE0pFUeP1P9HLt q2Qorm8lofnfAZSexjvqHIj3TAYJvtSZz8+EHlK8O2wm+bsWjr91/hHNl3YEaN4s5vJOWyLf 2D6B+YbpkIQX7d9TvDhdRfOGKT/mn/f20l8t2yP9NFPQqPME3bpN+6WqFs8Yebh4Y4G/sYwq QkNJBhTHcGwKZ2wept+x39tBxZhiV3NDQxEixgnscs5pfCIxIClDsAPvcYO1w8iAGGYBm8F1 W9JiMyS7kmurvIZjLGM/4Z4a7eit832uqbXzjScumgcqPG9yOZvK+ZvqyJiTY+vjOF+Fl3i7 sJj7yTZEnkQyM5pnR3K1Ni9LqdakJKkKteqCpK+zs0QU/S7r0dm9LjTt29mFWAYp4mWt7rUq uUSZpy/M6kIcQygSZHbnGpVclqks/FbQZe/T5WoEfRdaypCKRbKPw/mZcvYbZY5wSBAOC7p3 XczELSlCxvb8rczclv5IguGyx5p7SuyOLGs4PehyXa1dta3v99QaQ5qY9l3gZrpZNL18wfbt 1udswLbaFXTJzi+6e3fh5hufb1ttci2vCuy5tS5euf1AMDhvr+KIP8O96IO12YdSKrNvP913 RpNf9Vnhn8SmcyXlBekZm7/c0WKeqZdV0lcOKki9SpmcSOj0yv8Bb5O8i1kDAAA= X-Brightmail-Tracker: H4sIAAAAAAAAAzWSfUhTexjH7++8/M5xtTqsUYfiUi2ke40sK+uh25UL/eGvlxsRRBSEHurQ VmqyqWURaJp49WovYJapzRnL5krbDOxlc7hcLcks1zQxS6mR5UuUW9e3vLPL/efhw/f78Hn+ eXhadYNdyOtS0mR9ipSkwQpGseO3nJWp36K0q290zIXgaD4D5XVWDO23ahFYG7IpGGiJh87Q IIKJp89oKC1pR1DV95qGBk8vAkfNaQwd7+aALziCwVtSiCGnug7D80+TFPRcvEBBre1PeGMO MNB6zkRB6QCGK6U5VHh8oGDMbOHAnBUJ/TVlHEz2xYC318+Cu8LLgqN7BVyu7MHwwOFlwNPY T0HHvXIMvdZpFlo9jxkIFS+C9vNFLNwcNmH4FDLTYA6OcPDCZaTAY5wP9blha97X7yw8KnJR kHftNgW+V/cROPPfUmCz+jG4g4MU2G0lNIxfb0HQXzzEwZm/xzi4kl2MoPDMRQZye2Jh4p/w 5YrRGMi+Ws/AzSk/+iOOWCutiLgHR2iSaz9GxoMvMXGEjAx5YhLJ3bLXHMl1dnPEaEsn9poo Uv1ggCJVX4IssVn+wsT25QJHCoZ8FBlua+N2/rxPsemgnKTLkPWr4hIV2jp3H5OavfG4rzoP Z6Gu6AIUwYvCOtHnceIZxsJysatrjJ5htbBEtBcF2AKk4GnBP0vsrHiFChDPzxP2ig9Nv8/s MEKk2HD2FjXDSmG9+LHIgv5zLhZr610/PBHhvLvQ/SNXCbGir7aSOYcURvSTBal1KRnJki4p NtpwRJuZojsefeBosg2FH8h8avJ8IxrtiG9GAo80s5X1jl+1KlbKMGQmNyORpzVqpcX+i1al PChlnpD1RxP06UmyoRkt4hnNAuXWPXKiSjgkpclHZDlV1v/fUnzEwiy0ZmjJB/+zoem1n0/2 VG3eNa4sQZ/3lxvUzcu46theJmFzU/67Jtd00+OVfTUPAw7v7CnvnTnzcsoDSyXV2gPbNm1o jdwdiBAOv49rce1XvkS6/J3xg5daG9XHJhwJyRmmzvStV9vStq9ednJLonOdtM9T5owcfits xI1zQ21k6oSGMWilmChab5D+BZdIMXw8AwAA X-CFilter-Loop: Reflected Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" dma fence can be used at various points in the code and it's very hard to distinguish dma fences between different usages. Using a single dept_key for all the dma fences could trigger false positive reports. Assign unique dept_key to each distinct dma fence wait to avoid false positive reports. Signed-off-by: Byungchul Park --- drivers/dma-buf/dma-fence.c | 18 ++++----- include/linux/dma-fence.h | 74 +++++++++++++++++++++++++++++-------- 2 files changed, 68 insertions(+), 24 deletions(-) diff --git a/drivers/dma-buf/dma-fence.c b/drivers/dma-buf/dma-fence.c index a45e5416f2dd..f6a26e9bbe0e 100644 --- a/drivers/dma-buf/dma-fence.c +++ b/drivers/dma-buf/dma-fence.c @@ -481,7 +481,7 @@ int dma_fence_signal(struct dma_fence *fence) EXPORT_SYMBOL(dma_fence_signal); =20 /** - * dma_fence_wait_timeout - sleep until the fence gets signaled + * __dma_fence_wait_timeout - sleep until the fence gets signaled * or until timeout elapses * @fence: the fence to wait on * @intr: if true, do an interruptible wait @@ -499,7 +499,7 @@ EXPORT_SYMBOL(dma_fence_signal); * See also dma_fence_wait() and dma_fence_wait_any_timeout(). */ signed long -dma_fence_wait_timeout(struct dma_fence *fence, bool intr, signed long tim= eout) +__dma_fence_wait_timeout(struct dma_fence *fence, bool intr, signed long t= imeout) { signed long ret; =20 @@ -520,7 +520,7 @@ dma_fence_wait_timeout(struct dma_fence *fence, bool in= tr, signed long timeout) trace_dma_fence_wait_end(fence); return ret; } -EXPORT_SYMBOL(dma_fence_wait_timeout); +EXPORT_SYMBOL(__dma_fence_wait_timeout); =20 /** * dma_fence_release - default release function for fences @@ -747,7 +747,7 @@ dma_fence_default_wait_cb(struct dma_fence *fence, stru= ct dma_fence_cb *cb) } =20 /** - * dma_fence_default_wait - default sleep until the fence gets signaled + * __dma_fence_default_wait - default sleep until the fence gets signaled * or until timeout elapses * @fence: the fence to wait on * @intr: if true, do an interruptible wait @@ -759,7 +759,7 @@ dma_fence_default_wait_cb(struct dma_fence *fence, stru= ct dma_fence_cb *cb) * functions taking a jiffies timeout. */ signed long -dma_fence_default_wait(struct dma_fence *fence, bool intr, signed long tim= eout) +__dma_fence_default_wait(struct dma_fence *fence, bool intr, signed long t= imeout) { struct default_wait_cb cb; unsigned long flags; @@ -808,7 +808,7 @@ dma_fence_default_wait(struct dma_fence *fence, bool in= tr, signed long timeout) spin_unlock_irqrestore(fence->lock, flags); return ret; } -EXPORT_SYMBOL(dma_fence_default_wait); +EXPORT_SYMBOL(__dma_fence_default_wait); =20 static bool dma_fence_test_signaled_any(struct dma_fence **fences, uint32_t count, @@ -828,7 +828,7 @@ dma_fence_test_signaled_any(struct dma_fence **fences, = uint32_t count, } =20 /** - * dma_fence_wait_any_timeout - sleep until any fence gets signaled + * __dma_fence_wait_any_timeout - sleep until any fence gets signaled * or until timeout elapses * @fences: array of fences to wait on * @count: number of fences to wait on @@ -848,7 +848,7 @@ dma_fence_test_signaled_any(struct dma_fence **fences, = uint32_t count, * See also dma_fence_wait() and dma_fence_wait_timeout(). */ signed long -dma_fence_wait_any_timeout(struct dma_fence **fences, uint32_t count, +__dma_fence_wait_any_timeout(struct dma_fence **fences, uint32_t count, bool intr, signed long timeout, uint32_t *idx) { struct default_wait_cb *cb; @@ -916,7 +916,7 @@ dma_fence_wait_any_timeout(struct dma_fence **fences, u= int32_t count, =20 return ret; } -EXPORT_SYMBOL(dma_fence_wait_any_timeout); +EXPORT_SYMBOL(__dma_fence_wait_any_timeout); =20 /** * DOC: deadline hints diff --git a/include/linux/dma-fence.h b/include/linux/dma-fence.h index e7ad819962e3..d7dc7dcd213f 100644 --- a/include/linux/dma-fence.h +++ b/include/linux/dma-fence.h @@ -393,8 +393,22 @@ int dma_fence_signal_locked(struct dma_fence *fence); int dma_fence_signal_timestamp(struct dma_fence *fence, ktime_t timestamp); int dma_fence_signal_timestamp_locked(struct dma_fence *fence, ktime_t timestamp); -signed long dma_fence_default_wait(struct dma_fence *fence, +signed long __dma_fence_default_wait(struct dma_fence *fence, bool intr, signed long timeout); + +/* + * Associate every caller with its own dept map. + */ +#define dma_fence_default_wait(f, intr, t) \ +({ \ + signed long __ret; \ + \ + sdt_might_sleep_start_timeout(NULL, t); \ + __ret =3D __dma_fence_default_wait(f, intr, t); \ + sdt_might_sleep_end(); \ + __ret; \ +}) + int dma_fence_add_callback(struct dma_fence *fence, struct dma_fence_cb *cb, dma_fence_func_t func); @@ -609,12 +623,37 @@ static inline ktime_t dma_fence_timestamp(struct dma_= fence *fence) return fence->timestamp; } =20 -signed long dma_fence_wait_timeout(struct dma_fence *, +signed long __dma_fence_wait_timeout(struct dma_fence *, bool intr, signed long timeout); -signed long dma_fence_wait_any_timeout(struct dma_fence **fences, +signed long __dma_fence_wait_any_timeout(struct dma_fence **fences, uint32_t count, bool intr, signed long timeout, uint32_t *idx); +/* + * Associate every caller with its own dept map. + */ +#define dma_fence_wait_timeout(f, intr, t) \ +({ \ + signed long __ret; \ + \ + sdt_might_sleep_start_timeout(NULL, t); \ + __ret =3D __dma_fence_wait_timeout(f, intr, t); \ + sdt_might_sleep_end(); \ + __ret; \ +}) + +/* + * Associate every caller with its own dept map. + */ +#define dma_fence_wait_any_timeout(fpp, count, intr, t, idx) \ +({ \ + signed long __ret; \ + \ + sdt_might_sleep_start_timeout(NULL, t); \ + __ret =3D __dma_fence_wait_any_timeout(fpp, count, intr, t, idx); \ + sdt_might_sleep_end(); \ + __ret; \ +}) =20 /** * dma_fence_wait - sleep until the fence gets signaled @@ -630,19 +669,24 @@ signed long dma_fence_wait_any_timeout(struct dma_fen= ce **fences, * fence might be freed before return, resulting in undefined behavior. * * See also dma_fence_wait_timeout() and dma_fence_wait_any_timeout(). + * + * Associate every caller with its own dept map. */ -static inline signed long dma_fence_wait(struct dma_fence *fence, bool int= r) -{ - signed long ret; - - /* Since dma_fence_wait_timeout cannot timeout with - * MAX_SCHEDULE_TIMEOUT, only valid return values are - * -ERESTARTSYS and MAX_SCHEDULE_TIMEOUT. - */ - ret =3D dma_fence_wait_timeout(fence, intr, MAX_SCHEDULE_TIMEOUT); - - return ret < 0 ? ret : 0; -} +#define dma_fence_wait(f, intr) \ +({ \ + signed long __ret; \ + \ + sdt_might_sleep_start_timeout(NULL, MAX_SCHEDULE_TIMEOUT); \ + __ret =3D __dma_fence_wait_timeout(f, intr, MAX_SCHEDULE_TIMEOUT);\ + sdt_might_sleep_end(); \ + \ + /* \ + * Since dma_fence_wait_timeout cannot timeout with \ + * MAX_SCHEDULE_TIMEOUT, only valid return values are \ + * -ERESTARTSYS and MAX_SCHEDULE_TIMEOUT. \ + */ \ + __ret < 0 ? __ret : 0; \ +}) =20 void dma_fence_set_deadline(struct dma_fence *fence, ktime_t deadline); =20 --=20 2.17.1 From nobody Fri Dec 19 15:51:27 2025 Received: from invmail4.hynix.com (exvmail4.hynix.com [166.125.252.92]) by smtp.subspace.kernel.org (Postfix) with ESMTP id B775A27F16F; Mon, 19 May 2025 09:19:00 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=166.125.252.92 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747646344; cv=none; b=gZxJE5JaBNGI878e0XBsLcXsCuVZR8G1690Vo9jmcZ2+OVgInDNVrM1TZih3jX9ZiZqUEAsPynmvAEUA7Asd3mJBk6KcGKuYYxoA7n60cDSpB8ztUa/z/nQ/5dD3vJjShSbfI4sOvm4A67k2PjNbCOVCGzh1isAB6T1l+59ALPk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747646344; c=relaxed/simple; bh=96XOFp5m0WCi1dCnE/wsAjz4TPWG2W3rMJRjru4SmyI=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References; b=kpE/U/tEedGiUnIvc/XzZlI9JOZLtyqWrbzdL+zC2ZvvbnZVmFjxG+StiS+byrfBEAgTa0NwgWAdQUbe6SiPAD5z5MdepZPGt6eOK6EcKUS5HcysWzI7w8W0TW5iQC+S1cbOQFjhV760exPj9qFgxhKVPvN8yXqIY78BelmXQ2k= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com; spf=pass smtp.mailfrom=sk.com; arc=none smtp.client-ip=166.125.252.92 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=sk.com X-AuditID: a67dfc5b-681ff7000002311f-d8-682af770b4b4 From: Byungchul Park To: linux-kernel@vger.kernel.org Cc: kernel_team@skhynix.com, torvalds@linux-foundation.org, damien.lemoal@opensource.wdc.com, linux-ide@vger.kernel.org, adilger.kernel@dilger.ca, linux-ext4@vger.kernel.org, mingo@redhat.com, peterz@infradead.org, will@kernel.org, tglx@linutronix.de, rostedt@goodmis.org, joel@joelfernandes.org, sashal@kernel.org, daniel.vetter@ffwll.ch, duyuyang@gmail.com, johannes.berg@intel.com, tj@kernel.org, tytso@mit.edu, willy@infradead.org, david@fromorbit.com, amir73il@gmail.com, gregkh@linuxfoundation.org, kernel-team@lge.com, linux-mm@kvack.org, akpm@linux-foundation.org, mhocko@kernel.org, minchan@kernel.org, hannes@cmpxchg.org, vdavydov.dev@gmail.com, sj@kernel.org, jglisse@redhat.com, dennis@kernel.org, cl@linux.com, penberg@kernel.org, rientjes@google.com, vbabka@suse.cz, ngupta@vflare.org, linux-block@vger.kernel.org, josef@toxicpanda.com, linux-fsdevel@vger.kernel.org, jack@suse.cz, jlayton@kernel.org, dan.j.williams@intel.com, hch@infradead.org, djwong@kernel.org, dri-devel@lists.freedesktop.org, rodrigosiqueiramelo@gmail.com, melissa.srw@gmail.com, hamohammed.sa@gmail.com, harry.yoo@oracle.com, chris.p.wilson@intel.com, gwan-gyeong.mun@intel.com, max.byungchul.park@gmail.com, boqun.feng@gmail.com, longman@redhat.com, yskelg@gmail.com, yunseong.kim@ericsson.com, yeoreum.yun@arm.com, netdev@vger.kernel.org, matthew.brost@intel.com, her0gyugyu@gmail.com Subject: [PATCH v16 33/42] dept: make dept aware of lockdep_set_lock_cmp_fn() annotation Date: Mon, 19 May 2025 18:18:17 +0900 Message-Id: <20250519091826.19752-34-byungchul@sk.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20250519091826.19752-1-byungchul@sk.com> References: <20250519091826.19752-1-byungchul@sk.com> X-Brightmail-Tracker: H4sIAAAAAAAAAzWSW0xTaRDH/c7lO4dqzdku6lE3izZhVTZeMGImUdl9Mftll41GJTH6oFVO bCM3C3IxMaGC2i2CrHJbBCzFHLFUxZYooiVAFUSDVoWCpKISY8SCJGirSNUtGF8mv8zM/zcv w9OqTnYBr0tOl/TJmkQ1VjCK0Vk1y1M/RGlXlbxZB/73RgYqL9swuC/VI7A1GigYvv0H9AVG EEx2P6ChrMSNoObFUxoaOwYROOuOYHj8cjb0+McwdJXkY8itvYzhoS9Igbf0FAX19r/hmfyK gXtFFgrKhjGcKculQuU1BROylQM5JxKG6io4CL6Ihq5BDwvOgV/hv2ovhpvOLgY6moYoeNxc iWHQ9pWFex13GAgULgT3vwUsXHxrweALyDTI/jEOHrWaKegwz4WGvJDw2LsvLHQWtFJw7NwV Cnqe3EDQYnxOgd3mweDyj1DgsJfQ8On8bQRDhaMcHD0xwcEZQyGC/KOlDOR5Y2DyY+hy1fto MJxtYODiZw/6fQOxVdsQcY2M0STPkUk++XsxcQbMDLlrEcn1iqccyWsZ4IjZfpA46qJI7c1h itSM+1lit/6DiX38FEdMoz0UeXv/Prf5px2K9QlSoi5D0q+M3a3Qlg/msqnlS7KqrC6cgyoj TCiMF4U1YpvXR33nTt8JboqxsETs75+gpzhcWCQ6Cl6xJqTgacEzU+yreoKmBj8K28UmS+80 M0KkWNxtDAV4XimsFe9c3fTNGSHWN7ROe8JC7YF81/S6SogRe+qrmSmnKJwOE2W5G38LzBfb 6vqZIqQ0oxlWpNIlZyRpdIlrVmizk3VZK/amJNlR6L3kw8GdTWjcvbUdCTxSz1I2OJdpVawm Iy07qR2JPK0OV1odS7UqZYIm+5CkT9mlP5gopbWjhTyjnqdcHchMUAn7NOnSfklKlfTfpxQf tiAHmVrif9nbHL9xBt9b9du+H9q6y3G6u+bPa7eKhZHXusasa+pNcbEDpqY9jeXF2TbN3A1s 8LpicW0w/+4B+uPLisj42j2+N38VxUwGIh4dz4w715wSuXbxFYOrL46UXg0fn2PdFjfpKTWe 3G7MUp0lW+yGnAveZeKB0x9ij1v2O3+W1UyaVhMdRevTNP8DAQpVeFoDAAA= X-Brightmail-Tracker: H4sIAAAAAAAAAzWSf0zMcRjH+3x/d5x9dxrfMcNZi/xsE89E/IG+FIU/0IZu+s7dVHKXlNHK dXZKTU0/6Icrds51/XCHha61jkssoqTfFDtSNHRHSVyZf5699ryf5/X88zC45CY5h1HExAnK GFmUlBIRop0B6uWxP3zlq7QFHDhHtAQUVpooaK4oQ2C6nYLBwKMgeO0aQvCr6TkOeTnNCEr6 enC4be9FYDWcpaDl/QxodQ5T0JiTToH6WiUFLwbHMejOzcagzLwD3ugdBDy9WIpB3gAFBXlq zF0+YjCqN9KgT/aGfsMVGsb7/KCxt40EW1EjCdbOpXC5uJuCGmsjAfbqfgxa7hdS0Gv6Q8JT +2MCXJlzoTkrg4TyL6UUDLr0OOidwzS8rNNhYNfNgqpUt/Xc9wkSGjLqMDh3/RYGrR0PENRq 32JgNrVRYHMOYWAx5+AwduMRgv7MzzRoLozSUJCSiSBdk0tAarc//Prpvlw04gcpV6sIKP/d hjYF8qZiE+JtQ8M4n2o5yY85X1G81aUj+CelHH/vSg/Np9Z20rzOfIK3GHz5azUDGF/yzUny ZuN5ijd/y6b5tM+tGP/l2TM6bF64aH2kEKWIF5QrAyNE8vxeNRmb75NQZLRRyahwfhryZDh2 NdcweIGeZIr14drbR/FJ9mIXcJYMB5mGRAzOtk3jXhd1oMlgJruPqy59NcUE681datK6FxhG zK7hHt8N/eecz5VV1U15PN3tznTb1LiE9eday4qJi0ikQx5G5KWIiY+WKaL8V6iOyhNjFAkr Dh+LNiP3A+nPjGdVo5GWoHrEMkg6XVxlXSKXkLJ4VWJ0PeIYXOolNloWyyXiSFniKUF57JDy RJSgqkdzGUI6W7x9rxAhYY/I4oSjghArKP+nGOM5JxkFb/bY5VO3hzFpsiO8A+4z9gPH54k6 dnSpFzmR77Kur8MVW2rbsXfrloX+mdjY44i0ZHW81YwlPUxezR5vyWlgTksfnG33qOgqNGTu 3+C/7YDLGhdcc4qmtUGVCzUGmTYpLMw+IdrtsJeEircOhNjupHxtcoSs/bDHue2gsS/8k5RQ yWV+vrhSJfsLLmraaTwDAAA= X-CFilter-Loop: Reflected Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" commit eb1cfd09f788e (lockdep: Add lock_set_cmp_fn() annotation) has been added to address the issue that lockdep was not able to detect a true deadlock like the following: https://lore.kernel.org/lkml/20220510232633.GA18445@X58A-UD3R/ The approach is only for lockdep but dept should work being aware of it because the new annotation is already used to avoid false positive of lockdep in the code. Make dept aware of the new lockdep annotation. Signed-off-by: Byungchul Park --- include/linux/dept.h | 10 +++++++++ kernel/dependency/dept.c | 48 +++++++++++++++++++++++++++++++++++----- kernel/locking/lockdep.c | 1 + 3 files changed, 53 insertions(+), 6 deletions(-) diff --git a/include/linux/dept.h b/include/linux/dept.h index b6dc4ff19537..8b4d97fc4627 100644 --- a/include/linux/dept.h +++ b/include/linux/dept.h @@ -130,6 +130,11 @@ struct dept_map { const char *name; struct dept_key *keys; =20 + /* + * keep lockdep map to handle lockdep_set_lock_cmp_fn(). + */ + void *lockdep_map; + /* * subclass that can be set from user */ @@ -156,6 +161,7 @@ struct dept_map { { \ .name =3D #n, \ .keys =3D (struct dept_key *)(k), \ + .lockdep_map =3D NULL, \ .sub_u =3D 0, \ .map_key =3D { .classes =3D { NULL, } }, \ .wgen =3D 0U, \ @@ -427,6 +433,8 @@ extern void dept_softirqs_on_ip(unsigned long ip); extern void dept_hardirqs_on(void); extern void dept_softirqs_off(void); extern void dept_hardirqs_off(void); + +#define dept_set_lockdep_map(m, lockdep_m) ({ (m)->lockdep_map =3D lockdep= _m; }) #else /* !CONFIG_DEPT */ struct dept_key { }; struct dept_map { }; @@ -469,5 +477,7 @@ struct dept_ext_wgen { }; #define dept_hardirqs_on() do { } while (0) #define dept_softirqs_off() do { } while (0) #define dept_hardirqs_off() do { } while (0) + +#define dept_set_lockdep_map(m, lockdep_m) do { } while (0) #endif #endif /* __LINUX_DEPT_H */ diff --git a/kernel/dependency/dept.c b/kernel/dependency/dept.c index dc3effabfab4..b154c1bb4ee5 100644 --- a/kernel/dependency/dept.c +++ b/kernel/dependency/dept.c @@ -1615,9 +1615,39 @@ static int next_wgen(void) return atomic_inc_return(&wgen) ?: atomic_inc_return(&wgen); } =20 -static void add_wait(struct dept_class *c, unsigned long ip, - const char *w_fn, int sub_l, bool sched_sleep, - bool timeout) +/* + * XXX: This is a temporary patch needed until lockdep stops tracking + * dependency in wrong way. lockdep has added an annotation to specify + * a callback to determin whether the given lock aquisition order is + * okay or not in its own way. Even though dept is already working + * correctly with sub class on that issue, it needs to be aware of the + * annotation anyway. + */ +static bool lockdep_cmp_fn(struct dept_map *prev, struct dept_map *next) +{ + /* + * Assumes the cmp_fn thing comes from struct lockdep_map. + */ + struct lockdep_map *p_lock =3D (struct lockdep_map *)prev->lockdep_map; + struct lockdep_map *n_lock =3D (struct lockdep_map *)next->lockdep_map; + struct lock_class *p_class =3D p_lock ? p_lock->class_cache[0] : NULL; + struct lock_class *n_class =3D n_lock ? n_lock->class_cache[0] : NULL; + + if (!p_class || !n_class) + return false; + + if (p_class !=3D n_class) + return false; + + if (!p_class->cmp_fn) + return false; + + return p_class->cmp_fn(p_lock, n_lock) < 0; +} + +static void add_wait(struct dept_map *m, struct dept_class *c, + unsigned long ip, const char *w_fn, int sub_l, + bool sched_sleep, bool timeout) { struct dept_task *dt =3D dept_task(); struct dept_wait *w; @@ -1658,8 +1688,13 @@ static void add_wait(struct dept_class *c, unsigned = long ip, if (!eh->ecxt) continue; =20 - if (eh->ecxt->class !=3D c || eh->sub_l =3D=3D sub_l) - add_dep(eh->ecxt, w); + if (eh->ecxt->class =3D=3D c && eh->sub_l !=3D sub_l) + continue; + + if (i =3D=3D dt->ecxt_held_pos - 1 && lockdep_cmp_fn(eh->map, m)) + continue; + + add_dep(eh->ecxt, w); } =20 wg =3D next_wgen(); @@ -2145,6 +2180,7 @@ void dept_map_init(struct dept_map *m, struct dept_ke= y *k, int sub_u, m->name =3D n; m->wgen =3D 0U; m->nocheck =3D !valid_key(k); + m->lockdep_map =3D NULL; =20 dept_exit_recursive(flags); } @@ -2366,7 +2402,7 @@ static void __dept_wait(struct dept_map *m, unsigned = long w_f, if (!c) continue; =20 - add_wait(c, ip, w_fn, sub_l, sched_sleep, timeout); + add_wait(m, c, ip, w_fn, sub_l, sched_sleep, timeout); } } =20 diff --git a/kernel/locking/lockdep.c b/kernel/locking/lockdep.c index d2805ce250cb..acab023eb015 100644 --- a/kernel/locking/lockdep.c +++ b/kernel/locking/lockdep.c @@ -5036,6 +5036,7 @@ void lockdep_set_lock_cmp_fn(struct lockdep_map *lock= , lock_cmp_fn cmp_fn, class->print_fn =3D print_fn; } =20 + dept_set_lockdep_map(&lock->dmap, lock); lockdep_recursion_finish(); raw_local_irq_restore(flags); } --=20 2.17.1 From nobody Fri Dec 19 15:51:27 2025 Received: from invmail4.hynix.com (exvmail4.hynix.com [166.125.252.92]) by smtp.subspace.kernel.org (Postfix) with ESMTP id CF08426D4DB; Mon, 19 May 2025 09:19:01 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=166.125.252.92 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747646345; cv=none; b=W2JWpAWXABEafAZwWYQPHVU2UG6X49nMGm0bcDGfyU80pUUtwvO9bi5icW72NlJDeIU3CC9Ei3Z0A0KRAHgIVak4cxtYThUbEmdK4xkdGNfIZ8iyOPLin5gmRq8fdHQ3bj+fJ6G79rY2y8JcxIQPMfkcmq7aZ6TP8dN7TLsH4rM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747646345; c=relaxed/simple; bh=ZkFyExeh1KnVuSPGRbTV7HuyWfkqFgTaYfXkUw08ekE=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References; b=kTifskCpASFtCPU5B+5d3Xn/G+T2w/eQN3sxphqTtAqxYK3bxhiQF7Fr/FV9Dy8I+/HQFL/YyB/7y5+ppHpg/SnUGObUI7E/6LaGaU2J8mpNqKZKK4lfXJCKqgLszc6CTLoQlBHs6OH6fBjJ2Fku1CpA4fDiMoHpYaGGy+hW/20= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com; spf=pass smtp.mailfrom=sk.com; arc=none smtp.client-ip=166.125.252.92 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=sk.com X-AuditID: a67dfc5b-669ff7000002311f-e7-682af770348a From: Byungchul Park To: linux-kernel@vger.kernel.org Cc: kernel_team@skhynix.com, torvalds@linux-foundation.org, damien.lemoal@opensource.wdc.com, linux-ide@vger.kernel.org, adilger.kernel@dilger.ca, linux-ext4@vger.kernel.org, mingo@redhat.com, peterz@infradead.org, will@kernel.org, tglx@linutronix.de, rostedt@goodmis.org, joel@joelfernandes.org, sashal@kernel.org, daniel.vetter@ffwll.ch, duyuyang@gmail.com, johannes.berg@intel.com, tj@kernel.org, tytso@mit.edu, willy@infradead.org, david@fromorbit.com, amir73il@gmail.com, gregkh@linuxfoundation.org, kernel-team@lge.com, linux-mm@kvack.org, akpm@linux-foundation.org, mhocko@kernel.org, minchan@kernel.org, hannes@cmpxchg.org, vdavydov.dev@gmail.com, sj@kernel.org, jglisse@redhat.com, dennis@kernel.org, cl@linux.com, penberg@kernel.org, rientjes@google.com, vbabka@suse.cz, ngupta@vflare.org, linux-block@vger.kernel.org, josef@toxicpanda.com, linux-fsdevel@vger.kernel.org, jack@suse.cz, jlayton@kernel.org, dan.j.williams@intel.com, hch@infradead.org, djwong@kernel.org, dri-devel@lists.freedesktop.org, rodrigosiqueiramelo@gmail.com, melissa.srw@gmail.com, hamohammed.sa@gmail.com, harry.yoo@oracle.com, chris.p.wilson@intel.com, gwan-gyeong.mun@intel.com, max.byungchul.park@gmail.com, boqun.feng@gmail.com, longman@redhat.com, yskelg@gmail.com, yunseong.kim@ericsson.com, yeoreum.yun@arm.com, netdev@vger.kernel.org, matthew.brost@intel.com, her0gyugyu@gmail.com Subject: [PATCH v16 34/42] dept: make dept stop from working on debug_locks_off() Date: Mon, 19 May 2025 18:18:18 +0900 Message-Id: <20250519091826.19752-35-byungchul@sk.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20250519091826.19752-1-byungchul@sk.com> References: <20250519091826.19752-1-byungchul@sk.com> X-Brightmail-Tracker: H4sIAAAAAAAAAzWSWUxTeRTG+d+dSic3dbsgZpwmxhGjiII5JqMxPuj1wcTlwUQn4zRytdVS sWWdiRlQIFAWHSISELBUU5tSEG95wKUEQYqViBVqLQRQGjUiIA7Yjmxqi/py8sv58n3nezgM LuskoxiVJkXQahRqOSUhJOMRtevP/B+j3NhVxYD/Yz4BVTetFLga6hBYm7IxGOnYDc8DYwhm Hz/BobzMhaB2eBCHJscQArv5HAW9r34Ct3+CAmdZIQXnr92k4OnoHAYDl0sxqBP3wgvTGwK6 LhoxKB+h4Er5eSw43mIwbbLQYMpaDT5zJQ1zw3HgHPKQYO9fBxU1AxTcszsJcDT7MOi9U0XB kPULCV2OhwQESlaA699iEurfGykYDZhwMPknaOhpNWDgMCyDxpxgYN7UZxI6i1sxyLt+CwN3 310ELfkvMRCtHgra/WMY2MQyHGZudCDwlYzTkFs0TcOV7BIEhbmXCcgZSIDZT8HL1R/jIPtq IwH18x60YxtvrbEivn1sAudzbOn8jP8ZxdsDBoJ/ZOT425WDNJ/T0k/zBjGVt5lj+Gv3RjC+ dtJP8qKlgOLFyVKa14+7Mf59dze9L/qw5LdEQa1KE7Sx2/+UKN9VfiCSh6UZ870rs1D1Ij0K Zzg2npvNm6d+8JQ+Fwsxxa7hvN5pPMRL2FWcrfgNqUcSBmc9i7jn1X0oJCxm93Men2PBQLCr OYv+MxliKbuF+9JThX8L/Zmra2xd4PDgvr+wfcErYxM4d10NEQrl2PJwrmXQ+r1FJHff7CUu IqkBhVmQTKVJS1Ko1PEblJkaVcaGY6eTRBR8L9PZuSPNaNJ1sA2xDJJHSBvta5UyUpGmy0xq QxyDy5dILbZflTJpoiLzL0F7+qg2VS3o2tAKhpAvl24KpCfK2BOKFOGUICQL2h8qxoRHZaGj 1wl7lq7i7476PfPp1KqNkQVJYZJY99s1LbuTXx84MBP2T3TU1DZNfP2FzM5Tx5tjFaUdK0vG JgsN6gzjpz7XpV/O/Bd2yDnqPBHpS0XdYsrOPZsaHphnSyVbHvZsfb155uklnddYlL+fqVA9 Pnls6UjTH7sSxOQbTQ2CV7qsOfV3OaFTKuJicK1O8RVrPZl9WgMAAA== X-Brightmail-Tracker: H4sIAAAAAAAAAzWSfUzMcRzHfb+/p+ty9nMav7GG2zyF9KB8NoQZfmwM/9jM6OjH3XrQ7hJF VirLpYapJl2uh51ch9w183QtnTuSTpQr7YoaJuVKdRGFK/PPe6+933u/P/98RIT0BjVbpIxL EFRx8hgZLSbFO1anL4//HqAIGrfMA89wFglFt400NN2qRGCsTsPQY9sCrSN9CH41viSgIK8J QUlXBwHV9k4EloozNDR/mAYtnn4a6vOyaUgvu03Dq94xDK78SxgqTdvhnf4TCQ0XSjEU9NBw tSAde+UzhlG9gQF96gLorihkYKwrGOo7nRRYtfUUWNqXwpViFw2PLPUk2O91Y2h+UERDp/EP BQ32ZySM5M6Bpos5FNx0l9LQO6InQO/pZ+B1rQ6DXTcTqjK8q2eHflPwNKcWw9nyOxha3j5E UJP1HoPJ6KTB6unDYDblEfDzug1Bd+5XBjLPjzJwNS0XQXZmPgkZrjD49cN7WTscDGnXqki4 Oe5E6yN4Y7ER8da+foLPMB/nf3re0LxlREfyz0s5/n5hB8Nn1LQzvM50jDdXBPBlj3owXzLo oXiT4RzNmwYvMbzmawvm3Q4Hs9N/r3hNlBCjTBRUKyIixYovhQNkfJfkxHizfyrS+mqQj4hj V3JDmkw8wTS7iGtrGyUm2I+dx5lzPlEaJBYRrNOXa9W+RRPBDHYX5+y2TxZIdgFn0PymJljC hnN/XhcR/0bncpVVtZPs4/Xbs62TXSkbxrVUFpMXkFiHphiQnzIuMVaujAkLVEcrkuKUJwIP HY01Ie8D6VPGLt5Dw81b6hArQrKpkirLEoWUkieqk2LrECciZH4Sg3mxQiqJkiclC6qjB1TH YgR1HZojImWzJNv2CJFS9og8QYgWhHhB9T/FIp/ZqSizcbfOVxO4Kbzjo/+M6s3JobZlG5Nt Dsv16h8py4K+v48+GPx4E+MOLz/n6FrljtCcyXct7D2s7awNWbJirXTfyT0D4lPv1tVsbbyh YLQrD8jnS5+0Octxlj1oKMQWNXh51ovTGxa7pftddw/7JBxUh5bEJz5cFOng1kRt/madHm6U kWqFPDiAUKnlfwGM1sM4PAMAAA== X-CFilter-Loop: Reflected Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" For many reasons, debug_locks_off() is called to stop lock debuging feature e.g. on panic(). dept should also stop it in the conditions. Signed-off-by: Byungchul Park --- include/linux/dept.h | 2 ++ kernel/dependency/dept.c | 6 ++++++ lib/debug_locks.c | 2 ++ 3 files changed, 10 insertions(+) diff --git a/include/linux/dept.h b/include/linux/dept.h index 8b4d97fc4627..b164f74e86e5 100644 --- a/include/linux/dept.h +++ b/include/linux/dept.h @@ -390,6 +390,7 @@ struct dept_ext_wgen { unsigned int wgen; }; =20 +extern void dept_stop_emerg(void); extern void dept_on(void); extern void dept_off(void); extern void dept_init(void); @@ -442,6 +443,7 @@ struct dept_ext_wgen { }; =20 #define DEPT_MAP_INITIALIZER(n, k) { } =20 +#define dept_stop_emerg() do { } while (0) #define dept_on() do { } while (0) #define dept_off() do { } while (0) #define dept_init() do { } while (0) diff --git a/kernel/dependency/dept.c b/kernel/dependency/dept.c index b154c1bb4ee5..b5ba6d939932 100644 --- a/kernel/dependency/dept.c +++ b/kernel/dependency/dept.c @@ -187,6 +187,12 @@ static void dept_unlock(void) arch_spin_unlock(&dept_spin); } =20 +void dept_stop_emerg(void) +{ + WRITE_ONCE(dept_stop, 1); +} +EXPORT_SYMBOL_GPL(dept_stop_emerg); + enum bfs_ret { BFS_CONTINUE, BFS_DONE, diff --git a/lib/debug_locks.c b/lib/debug_locks.c index a75ee30b77cb..14a965914a8f 100644 --- a/lib/debug_locks.c +++ b/lib/debug_locks.c @@ -38,6 +38,8 @@ EXPORT_SYMBOL_GPL(debug_locks_silent); */ int debug_locks_off(void) { + dept_stop_emerg(); + if (debug_locks && __debug_locks_off()) { if (!debug_locks_silent) { console_verbose(); --=20 2.17.1 From nobody Fri Dec 19 15:51:27 2025 Received: from invmail4.hynix.com (exvmail4.hynix.com [166.125.252.92]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 5F1BB281376; Mon, 19 May 2025 09:19:03 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=166.125.252.92 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747646347; cv=none; b=dk8PkBAaPrKIU/pvSAPbYZsF83OG4a+aVUm7GQYKXjO1SK6GFmkY2m4w5KA3UYOhpzfdwbaMs/3UwchY5J9YYw7nCxMp0cxNj6WjzUNCDqgagRPmkgwM9o061wr8W//FSHsM1gQ+Sc4oZGM2ccTP1KCfKMohfhXUcadBw+qyofM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747646347; c=relaxed/simple; bh=oQEw6Eo1ZD4NTR5FBfSUcltAFBYsFRqKtAgptAn2JR8=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References; b=gvBMOU+RH46NM8cb6zEyz5W49tjUHLpHv7D8t4P6zGzB7CWtThUH/JXNbAsNBLOky6nnHq3QCr+PZczHWfzKZxMZd1Dmzy3xqfBZVNysg0RnUS2cC2qSz8UToKjaZHotSmcQtU3/E0BDU+C9g/SE6jo1J0oT+Ux78sGbqpLaS9Q= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com; spf=pass smtp.mailfrom=sk.com; arc=none smtp.client-ip=166.125.252.92 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=sk.com X-AuditID: a67dfc5b-669ff7000002311f-f6-682af77082b9 From: Byungchul Park To: linux-kernel@vger.kernel.org Cc: kernel_team@skhynix.com, torvalds@linux-foundation.org, damien.lemoal@opensource.wdc.com, linux-ide@vger.kernel.org, adilger.kernel@dilger.ca, linux-ext4@vger.kernel.org, mingo@redhat.com, peterz@infradead.org, will@kernel.org, tglx@linutronix.de, rostedt@goodmis.org, joel@joelfernandes.org, sashal@kernel.org, daniel.vetter@ffwll.ch, duyuyang@gmail.com, johannes.berg@intel.com, tj@kernel.org, tytso@mit.edu, willy@infradead.org, david@fromorbit.com, amir73il@gmail.com, gregkh@linuxfoundation.org, kernel-team@lge.com, linux-mm@kvack.org, akpm@linux-foundation.org, mhocko@kernel.org, minchan@kernel.org, hannes@cmpxchg.org, vdavydov.dev@gmail.com, sj@kernel.org, jglisse@redhat.com, dennis@kernel.org, cl@linux.com, penberg@kernel.org, rientjes@google.com, vbabka@suse.cz, ngupta@vflare.org, linux-block@vger.kernel.org, josef@toxicpanda.com, linux-fsdevel@vger.kernel.org, jack@suse.cz, jlayton@kernel.org, dan.j.williams@intel.com, hch@infradead.org, djwong@kernel.org, dri-devel@lists.freedesktop.org, rodrigosiqueiramelo@gmail.com, melissa.srw@gmail.com, hamohammed.sa@gmail.com, harry.yoo@oracle.com, chris.p.wilson@intel.com, gwan-gyeong.mun@intel.com, max.byungchul.park@gmail.com, boqun.feng@gmail.com, longman@redhat.com, yskelg@gmail.com, yunseong.kim@ericsson.com, yeoreum.yun@arm.com, netdev@vger.kernel.org, matthew.brost@intel.com, her0gyugyu@gmail.com Subject: [PATCH v16 35/42] i2c: rename wait_for_completion callback to wait_for_completion_cb Date: Mon, 19 May 2025 18:18:19 +0900 Message-Id: <20250519091826.19752-36-byungchul@sk.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20250519091826.19752-1-byungchul@sk.com> References: <20250519091826.19752-1-byungchul@sk.com> X-Brightmail-Tracker: H4sIAAAAAAAAAzWSe0xTdxTH/f3uq63UXTuid6hxaaIi22Qsak50GjXG3Swh2aYx6iKukRvb WSprFYpxC9jKEISgiKSKUIopCGXWW0zUUaw0gIypdRYoDNkgvnhHpBUUH4XNf04+Od/H+edI CMUtKkqi0R0U9DqVVknLSNlwRNlnP72IUX/eWBwBwfEsEoovOWjw/VaNwFGbgaG/8SvoCA0h eHX7LgFFhT4EZb0PCKht6kHgrjxKw/2Hc8AfHKWhpTCHBlP5JRruDU5h6D5zCkO1GA//2B+T 0Jpvw1DUT8O5IhMOj6cYJu1VDNjTl0Bf5VkGpnrjoKWnnQJ31ydgKemmoc7dQkLT1T4M968X 09DjeEtBa9MtEkJ5C8B3MpeCmhEbDYMhOwH24CgDf3msGJqs88BpDhdmPn9DQXOuB0PmhcsY /J2/I6jP+heD6GinwRscwuASCwl4WdGIoC9vmIFjJyYZOJeRhyDn2BkSzN2r4NVE+PL58TjI KHWSUPO6HW1YxztKHIj3Do0SvNmVyr8MttG8O2Ql+T9sHH/t7AOGN9d3MbxVPMS7KmP48rp+ zJeNBSlerDpO8+LYKYbPHvZjfuTOHeabhbtkXyYKWk2KoI9d/4NMPdjvoJJrOGOzw4bTUW9k NpJKOHYlZ2q8iN9zqb+QmGaaXcYFApMzHMl+zLlyH1PZSCYh2PbZXMf5TjQtfMju5p7da54x kewSznv0LjPNcnY198RX+n/pYq7a6ZnxSMP7rhzvTFbBruL81SXkdCnHFkg5598m+r/AR9zN ygCZj+RWNKsKKTS6lCSVRrtyhTpNpzGu2HsgSUTh/7L/PPX9VTTm29qAWAlSRsid7uVqBaVK MaQlNSBOQigj5VWuaLVCnqhKOyzoD+zRH9IKhga0QEIq58u/CKUmKth9qoPCfkFIFvTvVSyR RqUj3ZwjBZbAni1/fvDdpv3ygSO3PfPnbZ+wmKUDsRXFgYnoC22f3oiyJOfXfbvWrIK56R7D 6eePXpw8cX3HwrTD4rgm0r38SX18hrYu/lHW2skWG3NN9HYaC0Z/XfT6686tS38xrttmii2/ kjr75oaEvZk7jZsjsgZCP65xDz5rtWwcSZilJA1qVVwMoTeo3gE448iHWwMAAA== X-Brightmail-Tracker: H4sIAAAAAAAAAzWSa1CMcRTG/f/vtdWa11p51bgtTamRmhHHdXzr1cT44DLCZId37E4Xza4i l1HaXEqhma1Bl63MyraR3T6E1qQdZRlZ2lKpqDFIEWlTyWXL+HLmN+d5zvN8OSwhM1G+rDrh sKhJUMYpaAkp2bI2fVnijyBVaNOFAHAPnyOh4LaZBuetCgTm6jQMfY8i4NXIAIKfz54TkK93 Iijp6SKguqEbga38NA3N72aAyz1Ig0OfRUN62W0aXvRPYOjMy8VQYdkMb4zvSXh6qRRDfh8N 1/LTsWd8xDBmNDFgTPWH3vKrDEz0hIGju5UCe6GDAltHMFwp6qSh1uYgoaGmF0PzvQIaus1/ KHja8JiEkRw/cF7OpqDySykN/SNGAozuQQZe1hkwNBh8oErnST3z/TcFjdl1GM5cv4PB1X4f wYNzbzFYzK002N0DGKwWPQHjNx4h6M35zEDGhTEGrqXlIMjKyCNB1xkOP0c9zYXDYZBWXEVC 5a9WtHGDYC4yI8E+MEgIOusRYdzdQgu2EQMpPCnlhbtXuxhB96CDEQyWJMFaHiSU1fZhoWTI TQkW03lasAzlMkLmZxcWvjQ1MVvnRUvWHRDj1MmiZvmGfRJVf5+ZSqzkjzaaS3Eq6pFnIi+W 51bwxS49Mck0F8C3tY1NsZxbyFuz31OZSMISXOt0/lVhO5oUZnF7+W8vGqdMJOfP208/ZyZZ yq3kPziL8b/QBXxFVd2Ux8uz78iyT93KuHDeVVFEXkISA5pmQnJ1QnK8Uh0XHqKNVaUkqI+G 7D8Ub0GeDzKenLhcg4abI+oRxyKFt7TKtlQlo5TJ2pT4esSzhEIuNVkDVTLpAWXKMVFzKEaT FCdq65EfSyrmSCN3ivtk3EHlYTFWFBNFzX8Vs16+qUjTHcDUtjijt5/N3ePqmhv1y2f9sqKQ FnXwqa0Zvar9frMLZraN4ehF24jIJbsND98GPbayu6KW30zRB/oONeZdX7jjviQmdPXX8tCo e3PQ/BPbN6V9ynW9Hm+PmObw9ufVwfKSu8dXBa4pU2i3RCoX5Okc6YMto0k+SYvV3qPVsRcV pFalDAsiNFrlX+hP++A9AwAA X-CFilter-Loop: Reflected Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Functionally no change. This patch is a preparation for DEPT(DEPendency Tracker) to track dependencies related to a scheduler API, wait_for_completion(). Unfortunately, struct i2c_algo_pca_data has a callback member named wait_for_completion, that is the same as the scheduler API, which makes it hard to change the scheduler API to a macro form because of the ambiguity. Add a postfix _cb to the callback member to remove the ambiguity. Signed-off-by: Byungchul Park --- drivers/i2c/algos/i2c-algo-pca.c | 2 +- drivers/i2c/busses/i2c-pca-isa.c | 2 +- drivers/i2c/busses/i2c-pca-platform.c | 2 +- include/linux/i2c-algo-pca.h | 2 +- 4 files changed, 4 insertions(+), 4 deletions(-) diff --git a/drivers/i2c/algos/i2c-algo-pca.c b/drivers/i2c/algos/i2c-algo-= pca.c index 384af88e58ad..3647e4e19360 100644 --- a/drivers/i2c/algos/i2c-algo-pca.c +++ b/drivers/i2c/algos/i2c-algo-pca.c @@ -30,7 +30,7 @@ static int i2c_debug; #define pca_clock(adap) adap->i2c_clock #define pca_set_con(adap, val) pca_outw(adap, I2C_PCA_CON, val) #define pca_get_con(adap) pca_inw(adap, I2C_PCA_CON) -#define pca_wait(adap) adap->wait_for_completion(adap->data) +#define pca_wait(adap) adap->wait_for_completion_cb(adap->data) =20 static void pca_reset(struct i2c_algo_pca_data *adap) { diff --git a/drivers/i2c/busses/i2c-pca-isa.c b/drivers/i2c/busses/i2c-pca-= isa.c index 85e8cf58e8bf..0cbf2f509527 100644 --- a/drivers/i2c/busses/i2c-pca-isa.c +++ b/drivers/i2c/busses/i2c-pca-isa.c @@ -95,7 +95,7 @@ static struct i2c_algo_pca_data pca_isa_data =3D { /* .data intentionally left NULL, not needed with ISA */ .write_byte =3D pca_isa_writebyte, .read_byte =3D pca_isa_readbyte, - .wait_for_completion =3D pca_isa_waitforcompletion, + .wait_for_completion_cb =3D pca_isa_waitforcompletion, .reset_chip =3D pca_isa_resetchip, }; =20 diff --git a/drivers/i2c/busses/i2c-pca-platform.c b/drivers/i2c/busses/i2c= -pca-platform.c index 87da8241b927..c0f35ebbe37d 100644 --- a/drivers/i2c/busses/i2c-pca-platform.c +++ b/drivers/i2c/busses/i2c-pca-platform.c @@ -180,7 +180,7 @@ static int i2c_pca_pf_probe(struct platform_device *pde= v) } =20 i2c->algo_data.data =3D i2c; - i2c->algo_data.wait_for_completion =3D i2c_pca_pf_waitforcompletion; + i2c->algo_data.wait_for_completion_cb =3D i2c_pca_pf_waitforcompletion; if (i2c->gpio) i2c->algo_data.reset_chip =3D i2c_pca_pf_resetchip; else diff --git a/include/linux/i2c-algo-pca.h b/include/linux/i2c-algo-pca.h index 7c522fdd9ea7..e305bf32e40a 100644 --- a/include/linux/i2c-algo-pca.h +++ b/include/linux/i2c-algo-pca.h @@ -71,7 +71,7 @@ struct i2c_algo_pca_data { void *data; /* private low level data */ void (*write_byte) (void *data, int reg, int val); int (*read_byte) (void *data, int reg); - int (*wait_for_completion) (void *data); + int (*wait_for_completion_cb) (void *data); void (*reset_chip) (void *data); /* For PCA9564, use one of the predefined frequencies: * 330000, 288000, 217000, 146000, 88000, 59000, 44000, 36000 --=20 2.17.1 From nobody Fri Dec 19 15:51:27 2025 Received: from invmail4.hynix.com (exvmail4.skhynix.com [166.125.252.92]) by smtp.subspace.kernel.org (Postfix) with ESMTP id B3DC5281524; Mon, 19 May 2025 09:19:03 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=166.125.252.92 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747646348; cv=none; b=jlJvgaOBrw/b1Y7WVgW83aDrhrBPvBsk1nUzJq0Z0Zm5qBXzqvjZhiNMenpFKCsyDVyoZbLF3UQHp0EBWPhRdWkfOd9b6/iij2A5xQb0qaZIBgRqiFkRcb7BLS8pYXr/zSKXUiw0ObwfsgJ8GiGTLxwLqsJvl6pWKN+nkizZhSs= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747646348; c=relaxed/simple; bh=YrWBIIAMqnF09k3CiHle1jkWmEk31GBWH5LYnj+OH1k=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References; b=CqA5D5BRLyBiafDWoFt3IKuE73yP36YigYhTZJFWA/FExFrh34yYv7R2ql5snnXUkoUoIw+Be/xSI8MGKNK9kt9NbcA0KFUpVBGaR3BOq/7gNFY1+CNVFyPwc9/x2X9nUbYrAwcJ9yVdBJr4CP5w9IN3LKr+oYqBt4HVRRjJjEQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com; spf=pass smtp.mailfrom=sk.com; arc=none smtp.client-ip=166.125.252.92 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=sk.com X-AuditID: a67dfc5b-669ff7000002311f-07-682af7708f43 From: Byungchul Park To: linux-kernel@vger.kernel.org Cc: kernel_team@skhynix.com, torvalds@linux-foundation.org, damien.lemoal@opensource.wdc.com, linux-ide@vger.kernel.org, adilger.kernel@dilger.ca, linux-ext4@vger.kernel.org, mingo@redhat.com, peterz@infradead.org, will@kernel.org, tglx@linutronix.de, rostedt@goodmis.org, joel@joelfernandes.org, sashal@kernel.org, daniel.vetter@ffwll.ch, duyuyang@gmail.com, johannes.berg@intel.com, tj@kernel.org, tytso@mit.edu, willy@infradead.org, david@fromorbit.com, amir73il@gmail.com, gregkh@linuxfoundation.org, kernel-team@lge.com, linux-mm@kvack.org, akpm@linux-foundation.org, mhocko@kernel.org, minchan@kernel.org, hannes@cmpxchg.org, vdavydov.dev@gmail.com, sj@kernel.org, jglisse@redhat.com, dennis@kernel.org, cl@linux.com, penberg@kernel.org, rientjes@google.com, vbabka@suse.cz, ngupta@vflare.org, linux-block@vger.kernel.org, josef@toxicpanda.com, linux-fsdevel@vger.kernel.org, jack@suse.cz, jlayton@kernel.org, dan.j.williams@intel.com, hch@infradead.org, djwong@kernel.org, dri-devel@lists.freedesktop.org, rodrigosiqueiramelo@gmail.com, melissa.srw@gmail.com, hamohammed.sa@gmail.com, harry.yoo@oracle.com, chris.p.wilson@intel.com, gwan-gyeong.mun@intel.com, max.byungchul.park@gmail.com, boqun.feng@gmail.com, longman@redhat.com, yskelg@gmail.com, yunseong.kim@ericsson.com, yeoreum.yun@arm.com, netdev@vger.kernel.org, matthew.brost@intel.com, her0gyugyu@gmail.com Subject: [PATCH v16 36/42] dept: assign unique dept_key to each distinct wait_for_completion() caller Date: Mon, 19 May 2025 18:18:20 +0900 Message-Id: <20250519091826.19752-37-byungchul@sk.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20250519091826.19752-1-byungchul@sk.com> References: <20250519091826.19752-1-byungchul@sk.com> X-Brightmail-Tracker: H4sIAAAAAAAAAzWSeUiTcRjH+723y9XLut4ySAZd2qVYPYFFRdQLEUX1R9gfNvKtLeeqaR5B oDnDNM0OtUxtWSyZU9fWYdbMtGxqh+XyXF5FZW1a2laaHZvRPw8fni/fz/PPw+ASKzmLUahi BLVKppRSIkLk9C1afPh7gHxZ/9hkcH1LJSC/3EBBU1kJAsPNJAz6H2+CVrcDwc9nL3DIzW5C cKX3DQ4367oQWIqPU9D8bhLYXIMU1GenU5B8tZyCl5/HMLDnnMWgxLQFunXvCWjMKsIgt5+C S7nJmGd8xGBEp6dBlzgX+orzaBjrDYL6rhYSLB2BcLHQTsF9Sz0BdRV9GDRX5lPQZfhDQmOd lQB3ph80nckgoXSgiILPbh0OOtcgDa+qtRjUaaeDUeMRnhj+TcKTjGoMTly7gYGt/R6CqtQe DEyGFgpqXQ4MzKZsHEavP0bQl+mkIeXUCA2XkjIRpKfkEKCxL4efPzyXC74FQdJlIwGlv1rQ 2tW8odCA+FrHIM5rzHH8qOs1xVvcWoJvKOL4u3lvaF5T1UHzWtMR3lwcwF+934/xV4ZcJG/S n6R409BZmk9z2jB+4PlzetvsMFFohKBUxArqpWv2iOTWznvYoSRVvCa5gUhEp8PSkA/DsSHc 2/JK+j/fdn+lvEyx87m2thHcy1NZf86c8Z5MQyIGZ1smcq0F7cgbTGEF7o7lOOllgp3LtVpr x1nMruCGG3vxf9I5XImxepx9PPuO9NrxroRdztlKCgmvlGMv+3AFuizyX2Em97C4jchCYi2a oEcShSo2SqZQhiyRJ6gU8Uv2HowyIc9/6Y6N7a5AQ007ahDLIKmv2GhZKJeQstjohKgaxDG4 dKpYb14gl4gjZAlHBfXBcPURpRBdg/wYQjpDHOyOi5Cw+2UxQqQgHBLU/1OM8ZmViArnbfR3 99m+7EyFms3U4IcbMY/C1YntK19W9OCjvj3hvvS0u7ueZQY27FEccIZZI+vTzm91BJ97vbP5 ov7WtQNkT1lnSt4ZfH1kx4UNygvDnyqZCr+BzXHOp8eqAuO39uZdPxq6rjs0vrPNf2hfVP6i afYH59rtIfTaYOP2nFXdjv1SIlouCwrA1dGyvxQramJbAwAA X-Brightmail-Tracker: H4sIAAAAAAAAAzWSa0xTdxjG/f/PlbriSW30RJnOMrxgvJAM9yYzxOs8MZHwwWTGeKHRo22A QlphYGKkUlFArkklQ8FSTEdKHdgaxEuVUEVuIpfKpQIOtpg1oN2QVhG8FBa/PPnlefI+75eH JWRWahmr1pwStRplooKWkJLYn7I2pLyLVG0eMm8A/9RFEq7W2mjo+qMGge2WHoP38R7oD0wg mHn6jIBSYxeCytFhAm41jyBwVp+joffvUHD7fTS0GvNoyKqqpaF7fBbD0OUSDDX2ffDS8oqE 9iIzhlIvDVdKs3BQ/sEwbbEyYMmMgLHqMgZmR6OgdaSPAld5KwVOz3r4rWKIhvvOVhKaG8Yw 9N69SsOI7TMF7c0tJAQKlkNXcT4FN96YaRgPWAiw+H0M9DSaMDSblkCdIdia/fYTBU/yGzFk X7+JwT14D8GDi39isNv6aHD5JzA47EYCPvz+GMFYwWsGzl+aZuCKvgBB3vnLJBiGomHmffBz +VQU6K/VkXDjYx/aFiPYKmxIcE34CMHg+FX44H9OC86AiRTazLxwp2yYEQwPPIxgsqcKjupI oeq+FwuVk35KsFtzaME+WcIIua/dWHjT2cnEfXtQsvW4mKhOE7WbYuIlqpYX93CKXpNuyGoj M1HhwVwUwvLcD3x94D96jmluDT8wME3MsZz7jnfkv6JykYQluL6FfH/5IJoLFnMif9t5jppj kovg+1tc8yzltvBv20eJ/0tX8jV1jfMcEvQ9ea75WxkXzbtrKsgiJDGhBVYkV2vSkpTqxOiN ugRVhkadvvFYcpIdBRdkOTNb3ICmevc0IY5Fim+kdc51KhmlTNNlJDUhniUUcqnVsVYlkx5X ZpwWtclHtamJoq4JLWdJxVLp3l/EeBl3UnlKTBDFFFH7NcVsyLJMlPwxyhedXXzpR7Mjb8df Ky80DK6I8/68PfRTnLuTrH+4dhdbktTzYtW49d+pnX6vQa+L3f/kzGR4TH/YAc+sM4c1qpQR 070dZ4/JT8QcdoStKuloTKj9/kjOAs2huwM7wxcdGA6rWF1Yv8gX3mTs8nS/kz8qyt5v2B0/ gwtDF54uoxSkTqWMiiS0OuUXaFdiZj0DAAA= X-CFilter-Loop: Reflected Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" wait_for_completion() can be used at various points in the code and it's very hard to distinguish wait_for_completion()s between different usages. Using a single dept_key for all the wait_for_completion()s could trigger false positive reports. Assign unique dept_key to each distinct wait_for_completion() caller to avoid false positive reports. Signed-off-by: Byungchul Park --- include/linux/completion.h | 100 +++++++++++++++++++++++++++++++------ kernel/sched/completion.c | 60 +++++++++++----------- 2 files changed, 115 insertions(+), 45 deletions(-) diff --git a/include/linux/completion.h b/include/linux/completion.h index 3200b741de28..4d8fb1d95c0a 100644 --- a/include/linux/completion.h +++ b/include/linux/completion.h @@ -27,12 +27,10 @@ struct completion { unsigned int done; struct swait_queue_head wait; - struct dept_map dmap; }; =20 #define init_completion(x) \ do { \ - sdt_map_init(&(x)->dmap); \ __init_completion(x); \ } while (0) =20 @@ -43,17 +41,14 @@ do { \ =20 static inline void complete_acquire(struct completion *x, long timeout) { - sdt_might_sleep_start_timeout(&x->dmap, timeout); } =20 static inline void complete_release(struct completion *x) { - sdt_might_sleep_end(); } =20 #define COMPLETION_INITIALIZER(work) \ - { 0, __SWAIT_QUEUE_HEAD_INITIALIZER((work).wait), \ - .dmap =3D DEPT_MAP_INITIALIZER(work, NULL), } + { 0, __SWAIT_QUEUE_HEAD_INITIALIZER((work).wait), } =20 #define COMPLETION_INITIALIZER_ONSTACK_MAP(work, map) \ (*({ init_completion_map(&(work), &(map)); &(work); })) @@ -119,18 +114,18 @@ static inline void reinit_completion(struct completio= n *x) x->done =3D 0; } =20 -extern void wait_for_completion(struct completion *); -extern void wait_for_completion_io(struct completion *); -extern int wait_for_completion_interruptible(struct completion *x); -extern int wait_for_completion_killable(struct completion *x); -extern int wait_for_completion_state(struct completion *x, unsigned int st= ate); -extern unsigned long wait_for_completion_timeout(struct completion *x, +extern void __wait_for_completion(struct completion *); +extern void __wait_for_completion_io(struct completion *); +extern int __wait_for_completion_interruptible(struct completion *x); +extern int __wait_for_completion_killable(struct completion *x); +extern int __wait_for_completion_state(struct completion *x, unsigned int = state); +extern unsigned long __wait_for_completion_timeout(struct completion *x, unsigned long timeout); -extern unsigned long wait_for_completion_io_timeout(struct completion *x, +extern unsigned long __wait_for_completion_io_timeout(struct completion *x, unsigned long timeout); -extern long wait_for_completion_interruptible_timeout( +extern long __wait_for_completion_interruptible_timeout( struct completion *x, unsigned long timeout); -extern long wait_for_completion_killable_timeout( +extern long __wait_for_completion_killable_timeout( struct completion *x, unsigned long timeout); extern bool try_wait_for_completion(struct completion *x); extern bool completion_done(struct completion *x); @@ -139,4 +134,79 @@ extern void complete(struct completion *); extern void complete_on_current_cpu(struct completion *x); extern void complete_all(struct completion *); =20 +#define wait_for_completion(x) \ +({ \ + sdt_might_sleep_start_timeout(NULL, -1L); \ + __wait_for_completion(x); \ + sdt_might_sleep_end(); \ +}) +#define wait_for_completion_io(x) \ +({ \ + sdt_might_sleep_start_timeout(NULL, -1L); \ + __wait_for_completion_io(x); \ + sdt_might_sleep_end(); \ +}) +#define wait_for_completion_interruptible(x) \ +({ \ + int __ret; \ + \ + sdt_might_sleep_start_timeout(NULL, -1L); \ + __ret =3D __wait_for_completion_interruptible(x); \ + sdt_might_sleep_end(); \ + __ret; \ +}) +#define wait_for_completion_killable(x) \ +({ \ + int __ret; \ + \ + sdt_might_sleep_start_timeout(NULL, -1L); \ + __ret =3D __wait_for_completion_killable(x); \ + sdt_might_sleep_end(); \ + __ret; \ +}) +#define wait_for_completion_state(x, s) \ +({ \ + int __ret; \ + \ + sdt_might_sleep_start_timeout(NULL, -1L); \ + __ret =3D __wait_for_completion_state(x, s); \ + sdt_might_sleep_end(); \ + __ret; \ +}) +#define wait_for_completion_timeout(x, t) \ +({ \ + unsigned long __ret; \ + \ + sdt_might_sleep_start_timeout(NULL, t); \ + __ret =3D __wait_for_completion_timeout(x, t); \ + sdt_might_sleep_end(); \ + __ret; \ +}) +#define wait_for_completion_io_timeout(x, t) \ +({ \ + unsigned long __ret; \ + \ + sdt_might_sleep_start_timeout(NULL, t); \ + __ret =3D __wait_for_completion_io_timeout(x, t); \ + sdt_might_sleep_end(); \ + __ret; \ +}) +#define wait_for_completion_interruptible_timeout(x, t) \ +({ \ + long __ret; \ + \ + sdt_might_sleep_start_timeout(NULL, t); \ + __ret =3D __wait_for_completion_interruptible_timeout(x, t); \ + sdt_might_sleep_end(); \ + __ret; \ +}) +#define wait_for_completion_killable_timeout(x, t) \ +({ \ + long __ret; \ + \ + sdt_might_sleep_start_timeout(NULL, t); \ + __ret =3D __wait_for_completion_killable_timeout(x, t); \ + sdt_might_sleep_end(); \ + __ret; \ +}) #endif diff --git a/kernel/sched/completion.c b/kernel/sched/completion.c index 499b1fee9dc1..247169b26d81 100644 --- a/kernel/sched/completion.c +++ b/kernel/sched/completion.c @@ -4,7 +4,7 @@ * Generic wait-for-completion handler; * * It differs from semaphores in that their default case is the opposite, - * wait_for_completion default blocks whereas semaphore default non-block.= The + * __wait_for_completion default blocks whereas semaphore default non-bloc= k. The * interface also makes it easy to 'complete' multiple waiting threads, * something which isn't entirely natural for semaphores. * @@ -37,7 +37,7 @@ void complete_on_current_cpu(struct completion *x) * This will wake up a single thread waiting on this completion. Threads w= ill be * awakened in the same order in which they were queued. * - * See also complete_all(), wait_for_completion() and related routines. + * See also complete_all(), __wait_for_completion() and related routines. * * If this function wakes up a task, it executes a full memory barrier bef= ore * accessing the task state. @@ -134,23 +134,23 @@ wait_for_common_io(struct completion *x, long timeout= , int state) } =20 /** - * wait_for_completion: - waits for completion of a task + * __wait_for_completion: - waits for completion of a task * @x: holds the state of this particular completion * * This waits to be signaled for completion of a specific task. It is NOT * interruptible and there is no timeout. * - * See also similar routines (i.e. wait_for_completion_timeout()) with tim= eout + * See also similar routines (i.e. __wait_for_completion_timeout()) with t= imeout * and interrupt capability. Also see complete(). */ -void __sched wait_for_completion(struct completion *x) +void __sched __wait_for_completion(struct completion *x) { wait_for_common(x, MAX_SCHEDULE_TIMEOUT, TASK_UNINTERRUPTIBLE); } -EXPORT_SYMBOL(wait_for_completion); +EXPORT_SYMBOL(__wait_for_completion); =20 /** - * wait_for_completion_timeout: - waits for completion of a task (w/timeou= t) + * __wait_for_completion_timeout: - waits for completion of a task (w/time= out) * @x: holds the state of this particular completion * @timeout: timeout value in jiffies * @@ -162,28 +162,28 @@ EXPORT_SYMBOL(wait_for_completion); * till timeout) if completed. */ unsigned long __sched -wait_for_completion_timeout(struct completion *x, unsigned long timeout) +__wait_for_completion_timeout(struct completion *x, unsigned long timeout) { return wait_for_common(x, timeout, TASK_UNINTERRUPTIBLE); } -EXPORT_SYMBOL(wait_for_completion_timeout); +EXPORT_SYMBOL(__wait_for_completion_timeout); =20 /** - * wait_for_completion_io: - waits for completion of a task + * __wait_for_completion_io: - waits for completion of a task * @x: holds the state of this particular completion * * This waits to be signaled for completion of a specific task. It is NOT * interruptible and there is no timeout. The caller is accounted as waiti= ng * for IO (which traditionally means blkio only). */ -void __sched wait_for_completion_io(struct completion *x) +void __sched __wait_for_completion_io(struct completion *x) { wait_for_common_io(x, MAX_SCHEDULE_TIMEOUT, TASK_UNINTERRUPTIBLE); } -EXPORT_SYMBOL(wait_for_completion_io); +EXPORT_SYMBOL(__wait_for_completion_io); =20 /** - * wait_for_completion_io_timeout: - waits for completion of a task (w/tim= eout) + * __wait_for_completion_io_timeout: - waits for completion of a task (w/t= imeout) * @x: holds the state of this particular completion * @timeout: timeout value in jiffies * @@ -196,14 +196,14 @@ EXPORT_SYMBOL(wait_for_completion_io); * till timeout) if completed. */ unsigned long __sched -wait_for_completion_io_timeout(struct completion *x, unsigned long timeout) +__wait_for_completion_io_timeout(struct completion *x, unsigned long timeo= ut) { return wait_for_common_io(x, timeout, TASK_UNINTERRUPTIBLE); } -EXPORT_SYMBOL(wait_for_completion_io_timeout); +EXPORT_SYMBOL(__wait_for_completion_io_timeout); =20 /** - * wait_for_completion_interruptible: - waits for completion of a task (w/= intr) + * __wait_for_completion_interruptible: - waits for completion of a task (= w/intr) * @x: holds the state of this particular completion * * This waits for completion of a specific task to be signaled. It is @@ -211,7 +211,7 @@ EXPORT_SYMBOL(wait_for_completion_io_timeout); * * Return: -ERESTARTSYS if interrupted, 0 if completed. */ -int __sched wait_for_completion_interruptible(struct completion *x) +int __sched __wait_for_completion_interruptible(struct completion *x) { long t =3D wait_for_common(x, MAX_SCHEDULE_TIMEOUT, TASK_INTERRUPTIBLE); =20 @@ -219,10 +219,10 @@ int __sched wait_for_completion_interruptible(struct = completion *x) return t; return 0; } -EXPORT_SYMBOL(wait_for_completion_interruptible); +EXPORT_SYMBOL(__wait_for_completion_interruptible); =20 /** - * wait_for_completion_interruptible_timeout: - waits for completion (w/(t= o,intr)) + * __wait_for_completion_interruptible_timeout: - waits for completion (w/= (to,intr)) * @x: holds the state of this particular completion * @timeout: timeout value in jiffies * @@ -233,15 +233,15 @@ EXPORT_SYMBOL(wait_for_completion_interruptible); * or number of jiffies left till timeout) if completed. */ long __sched -wait_for_completion_interruptible_timeout(struct completion *x, +__wait_for_completion_interruptible_timeout(struct completion *x, unsigned long timeout) { return wait_for_common(x, timeout, TASK_INTERRUPTIBLE); } -EXPORT_SYMBOL(wait_for_completion_interruptible_timeout); +EXPORT_SYMBOL(__wait_for_completion_interruptible_timeout); =20 /** - * wait_for_completion_killable: - waits for completion of a task (killabl= e) + * __wait_for_completion_killable: - waits for completion of a task (killa= ble) * @x: holds the state of this particular completion * * This waits to be signaled for completion of a specific task. It can be @@ -249,7 +249,7 @@ EXPORT_SYMBOL(wait_for_completion_interruptible_timeout= ); * * Return: -ERESTARTSYS if interrupted, 0 if completed. */ -int __sched wait_for_completion_killable(struct completion *x) +int __sched __wait_for_completion_killable(struct completion *x) { long t =3D wait_for_common(x, MAX_SCHEDULE_TIMEOUT, TASK_KILLABLE); =20 @@ -257,9 +257,9 @@ int __sched wait_for_completion_killable(struct complet= ion *x) return t; return 0; } -EXPORT_SYMBOL(wait_for_completion_killable); +EXPORT_SYMBOL(__wait_for_completion_killable); =20 -int __sched wait_for_completion_state(struct completion *x, unsigned int s= tate) +int __sched __wait_for_completion_state(struct completion *x, unsigned int= state) { long t =3D wait_for_common(x, MAX_SCHEDULE_TIMEOUT, state); =20 @@ -267,10 +267,10 @@ int __sched wait_for_completion_state(struct completi= on *x, unsigned int state) return t; return 0; } -EXPORT_SYMBOL(wait_for_completion_state); +EXPORT_SYMBOL(__wait_for_completion_state); =20 /** - * wait_for_completion_killable_timeout: - waits for completion of a task = (w/(to,killable)) + * __wait_for_completion_killable_timeout: - waits for completion of a tas= k (w/(to,killable)) * @x: holds the state of this particular completion * @timeout: timeout value in jiffies * @@ -282,12 +282,12 @@ EXPORT_SYMBOL(wait_for_completion_state); * or number of jiffies left till timeout) if completed. */ long __sched -wait_for_completion_killable_timeout(struct completion *x, +__wait_for_completion_killable_timeout(struct completion *x, unsigned long timeout) { return wait_for_common(x, timeout, TASK_KILLABLE); } -EXPORT_SYMBOL(wait_for_completion_killable_timeout); +EXPORT_SYMBOL(__wait_for_completion_killable_timeout); =20 /** * try_wait_for_completion - try to decrement a completion without blocking @@ -329,7 +329,7 @@ EXPORT_SYMBOL(try_wait_for_completion); * completion_done - Test to see if a completion has any waiters * @x: completion structure * - * Return: 0 if there are waiters (wait_for_completion() in progress) + * Return: 0 if there are waiters (__wait_for_completion() in progress) * 1 if there are no waiters. * * Note, this will always return true if complete_all() was called on @X. --=20 2.17.1 From nobody Fri Dec 19 15:51:27 2025 Received: from invmail4.hynix.com (exvmail4.hynix.com [166.125.252.92]) by smtp.subspace.kernel.org (Postfix) with ESMTP id D847B2820A9; Mon, 19 May 2025 09:19:03 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=166.125.252.92 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747646348; cv=none; b=DZR7UlHSJp+ROEoGhs1mjIejmJ0oQ42w1eKM4VIuujS0xoc92Uty29KRdz1N+Oe4Ly+eBzhyRcgQM8WK0QcEkxUWagpKNDxq68RvBQ4m+ecGnXZJaA0/n4yocng5+ej3QRf0qDcHPhHfcBzne7Q/SyW8TvVIdHbJtumtVIbyykQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747646348; c=relaxed/simple; bh=314TpUAkZxMvbV2pk3wlciVKMbaC2tgJL1bvCZ48Y0A=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References; b=cyRSs2pwTg+85vKrHC+sL5YtCdWtBEbioQJRBrcBGJ0bY6gq/ZK6CATwiGVqR2LWgYJQOGRCyXh1Gcl6pMScbyrvfnufpLj3lc1ZAynZ0T1EnsX+hcMVEmTu/ehECMR9sCaM7sWOPrBoehAE/DTjQSj+cRmOxLB2DfMoHJNPznk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com; spf=pass smtp.mailfrom=sk.com; arc=none smtp.client-ip=166.125.252.92 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=sk.com X-AuditID: a67dfc5b-669ff7000002311f-16-682af77048c5 From: Byungchul Park To: linux-kernel@vger.kernel.org Cc: kernel_team@skhynix.com, torvalds@linux-foundation.org, damien.lemoal@opensource.wdc.com, linux-ide@vger.kernel.org, adilger.kernel@dilger.ca, linux-ext4@vger.kernel.org, mingo@redhat.com, peterz@infradead.org, will@kernel.org, tglx@linutronix.de, rostedt@goodmis.org, joel@joelfernandes.org, sashal@kernel.org, daniel.vetter@ffwll.ch, duyuyang@gmail.com, johannes.berg@intel.com, tj@kernel.org, tytso@mit.edu, willy@infradead.org, david@fromorbit.com, amir73il@gmail.com, gregkh@linuxfoundation.org, kernel-team@lge.com, linux-mm@kvack.org, akpm@linux-foundation.org, mhocko@kernel.org, minchan@kernel.org, hannes@cmpxchg.org, vdavydov.dev@gmail.com, sj@kernel.org, jglisse@redhat.com, dennis@kernel.org, cl@linux.com, penberg@kernel.org, rientjes@google.com, vbabka@suse.cz, ngupta@vflare.org, linux-block@vger.kernel.org, josef@toxicpanda.com, linux-fsdevel@vger.kernel.org, jack@suse.cz, jlayton@kernel.org, dan.j.williams@intel.com, hch@infradead.org, djwong@kernel.org, dri-devel@lists.freedesktop.org, rodrigosiqueiramelo@gmail.com, melissa.srw@gmail.com, hamohammed.sa@gmail.com, harry.yoo@oracle.com, chris.p.wilson@intel.com, gwan-gyeong.mun@intel.com, max.byungchul.park@gmail.com, boqun.feng@gmail.com, longman@redhat.com, yskelg@gmail.com, yunseong.kim@ericsson.com, yeoreum.yun@arm.com, netdev@vger.kernel.org, matthew.brost@intel.com, her0gyugyu@gmail.com Subject: [PATCH v16 37/42] completion, dept: introduce init_completion_dmap() API Date: Mon, 19 May 2025 18:18:21 +0900 Message-Id: <20250519091826.19752-38-byungchul@sk.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20250519091826.19752-1-byungchul@sk.com> References: <20250519091826.19752-1-byungchul@sk.com> X-Brightmail-Tracker: H4sIAAAAAAAAAzWSa0hTYRjHe885O+dsOTmtsJNW1shKpYvR4ulKlw+90L0Iogs58tBGOm2a l0LQsjBNs4VKamtqLXEz11ah5cI0VxaZ6TITs5KyTM3QtvLSZXb58vDj9+f5P18elpQ5RL6s WhMjaDXKcDktoSR9XoXzDn8LUi1s++ADrq+pFBSUm2lovGZCYL6RTEB33Xp44e5FMPLkKQm5 2Y0ICt++IuGGowOBveQ4Dc3vvMHp6qehPjudhhPF5TQ86xkloD1HR4DJugleG7soeJxVREBu Nw35uScIz/hIwJCxlAFjUgB0luQxMPo2BOo7WkRgbwuGC/p2Gqrs9RQ4KjoJaL5dQEOH+ZcI HjseUuDO9IPGcxkiKPtcREOP20iC0dXPQFO1gQCHwQcsKZ7CU4M/RfAgo5qAU5evE+B8eQfB 3dQ3BFjNLTTUunoJsFmzSRi+WoegM7OPgZNnhhjIT85EkH4yh4KUdgWMfPdcvvg1BJIvWSgo +9GCVq/EZr0Z4drefhKn2OLwsOs5je1uA4UfFfG4Mu8Vg1PutjHYYD2CbSVBuLiqm8CFAy4R tpaeprF1QMfgtD4ngT83NDBbp+6WrAgTwtWxgnbBqlCJ6kp5Ph1lD4y/P5iFktCwfxoSszy3 mB+4lkf9Z1NlqmiMaW4O39o6RI7xJG4Gb8vo8ngJS3It4/kXF1+isWAit43P0ek8yyxLcQF8 j8lrTEu5Jfxt/Wn0t9OfN1mq//SIPb4tvfaPl3EK3mnS/7t7VszXVe36y1P4eyWtVBaSGtC4 UiRTa2IjlOrwxfNVCRp1/PwDkRFW5PkuY+Longo00LijBnEskntJLfZAlUykjI1OiKhBPEvK J0lLbXNVMmmYMuGooI3crz0SLkTXID+Wkk+WLnLHhcm4g8oY4ZAgRAna/ynBin2TkP6sr8/2 9lWD5yyB09+LM6bN3Fk9PX62aoJC8aFrKDDqVpN2JKBh3Vw/ZV7y0n365jlXavc/27O1c/yy pooy+FTMb8z1W375B716YqK3OtLZcNMeszlxb1Rc0d7Q4coNMWmInPWlUJO2pXXteTb9mDpI F/yNPMTjsHmHxcGhcaFVa+RUtEoZEkRqo5W/ARfZoblZAwAA X-Brightmail-Tracker: H4sIAAAAAAAAAzWSb0zMcRzHfX//O85+O43fksVtSA014jOa5ZEv5t8jhs0dftzpOrc7RTbW VSylnKwa6Vx/nOqOzl2MuNY6ylXSP0U7oZlpRZYuSuFinnz23uu9vd5PPhwpq6BDOLX2hKjX KjVyRkJJtq9PW677HqGK8llXgX80g4LrVXYG2u7YENirjQQMPN0EPWNDCH4+f0FCQV4bguL3 b0iobuhD4C5PZaDzw2zo8g8z4M3LYiCttIqB9sFJAnz5uQTYnNvgrfUjBc2mEgIKBhgoLEgj AucTAePWShasKYuhv/waC5Pvo8Hb102Dp8hLg7s3Eq6afQw8dnspaHjQT0BnzXUG+uy/aWhu eEbBWM58aLucTcPtLyUMDI5ZSbD6h1noqLMQ0GCZC470gPX8t180NGbXEXC+7C4BXa8fIajN eEeA097NgMc/RIDLmUfCxK2nCPpzPrNw7uI4C4XGHARZ5/IpSPfFwM8fgeWi0Wgw3nBQcHuq G8VtwHazHWHP0DCJ010n8YT/JYPdYxYKN5UI+OG1NyxOr+1lscWZiF3lEbj08QCBi0f8NHZW XmCwcySXxZmfuwj8pbWV3blgryT2sKhRJ4n6lRsUEtXNqkJG51526sk3E0pBE2GZKIgT+NWC 7WEGPZ0Zfqnw6tU4OZ2D+YWCK/tjgEs4ku+eKfQUvUbTxRx+l5Cfm0tlIo6j+MXCoG3WNJby a4Qa8wX0zxkm2Bx1fz1BAd6b5fnLZXyM0GUzUyYksaAZlShYrU1KUKo1MSsM8apkrfrUikPH E5wo8EDWM5OXH6DRzk31iOeQfJbU4V6mktHKJENyQj0SOFIeLK10hatk0sPK5NOi/vgBfaJG NNSj+RwlnyfdsltUyPijyhNivCjqRP3/luCCQlJQ2b7SGphqrFiEOgbV65q3tmjDr2hE0XO2 lPuti/TG3glODe+Ldigs8b4zvT0m3TrDCD958P69WNmxrwomairsSH5L56PGNOmc0EvhVR0f Nu4PzdLQSxJj98SZsGJXddOh2vaOirJ3+5ZEOs1iamjSji21xgN29ea9ce0hxpNr5ZRBpYyO IPUG5R/iiH4zPAMAAA== X-CFilter-Loop: Reflected Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Currently, dept uses dept's map embedded in task_struct to track dependencies related to wait_for_completion() and its family. So it doesn't need an explicit map basically. However, for those who want to set the maps with customized class or key, introduce a new API to use external maps. Signed-off-by: Byungchul Park --- include/linux/completion.h | 40 +++++++++++++++++++++----------------- 1 file changed, 22 insertions(+), 18 deletions(-) diff --git a/include/linux/completion.h b/include/linux/completion.h index 4d8fb1d95c0a..e50f7d9b4b97 100644 --- a/include/linux/completion.h +++ b/include/linux/completion.h @@ -27,17 +27,15 @@ struct completion { unsigned int done; struct swait_queue_head wait; + struct dept_map *dmap; }; =20 -#define init_completion(x) \ -do { \ - __init_completion(x); \ -} while (0) +#define init_completion(x) init_completion_dmap(x, NULL) =20 /* - * XXX: No use cases for now. Fill the body when needed. + * XXX: This usage using lockdep's map should be deprecated. */ -#define init_completion_map(x, m) init_completion(x) +#define init_completion_map(x, m) init_completion_dmap(x, NULL) =20 static inline void complete_acquire(struct completion *x, long timeout) { @@ -48,8 +46,11 @@ static inline void complete_release(struct completion *x) } =20 #define COMPLETION_INITIALIZER(work) \ - { 0, __SWAIT_QUEUE_HEAD_INITIALIZER((work).wait), } + { 0, __SWAIT_QUEUE_HEAD_INITIALIZER((work).wait), .dmap =3D NULL, } =20 +/* + * XXX: This usage using lockdep's map should be deprecated. + */ #define COMPLETION_INITIALIZER_ONSTACK_MAP(work, map) \ (*({ init_completion_map(&(work), &(map)); &(work); })) =20 @@ -90,15 +91,18 @@ static inline void complete_release(struct completion *= x) #endif =20 /** - * __init_completion - Initialize a dynamically allocated completion + * init_completion_dmap - Initialize a dynamically allocated completion * @x: pointer to completion structure that is to be initialized + * @dmap: pointer to external dept's map to be used as a separated map * * This inline function will initialize a dynamically created completion * structure. */ -static inline void __init_completion(struct completion *x) +static inline void init_completion_dmap(struct completion *x, + struct dept_map *dmap) { x->done =3D 0; + x->dmap =3D dmap; init_swait_queue_head(&x->wait); } =20 @@ -136,13 +140,13 @@ extern void complete_all(struct completion *); =20 #define wait_for_completion(x) \ ({ \ - sdt_might_sleep_start_timeout(NULL, -1L); \ + sdt_might_sleep_start_timeout((x)->dmap, -1L); \ __wait_for_completion(x); \ sdt_might_sleep_end(); \ }) #define wait_for_completion_io(x) \ ({ \ - sdt_might_sleep_start_timeout(NULL, -1L); \ + sdt_might_sleep_start_timeout((x)->dmap, -1L); \ __wait_for_completion_io(x); \ sdt_might_sleep_end(); \ }) @@ -150,7 +154,7 @@ extern void complete_all(struct completion *); ({ \ int __ret; \ \ - sdt_might_sleep_start_timeout(NULL, -1L); \ + sdt_might_sleep_start_timeout((x)->dmap, -1L); \ __ret =3D __wait_for_completion_interruptible(x); \ sdt_might_sleep_end(); \ __ret; \ @@ -159,7 +163,7 @@ extern void complete_all(struct completion *); ({ \ int __ret; \ \ - sdt_might_sleep_start_timeout(NULL, -1L); \ + sdt_might_sleep_start_timeout((x)->dmap, -1L); \ __ret =3D __wait_for_completion_killable(x); \ sdt_might_sleep_end(); \ __ret; \ @@ -168,7 +172,7 @@ extern void complete_all(struct completion *); ({ \ int __ret; \ \ - sdt_might_sleep_start_timeout(NULL, -1L); \ + sdt_might_sleep_start_timeout((x)->dmap, -1L); \ __ret =3D __wait_for_completion_state(x, s); \ sdt_might_sleep_end(); \ __ret; \ @@ -177,7 +181,7 @@ extern void complete_all(struct completion *); ({ \ unsigned long __ret; \ \ - sdt_might_sleep_start_timeout(NULL, t); \ + sdt_might_sleep_start_timeout((x)->dmap, t); \ __ret =3D __wait_for_completion_timeout(x, t); \ sdt_might_sleep_end(); \ __ret; \ @@ -186,7 +190,7 @@ extern void complete_all(struct completion *); ({ \ unsigned long __ret; \ \ - sdt_might_sleep_start_timeout(NULL, t); \ + sdt_might_sleep_start_timeout((x)->dmap, t); \ __ret =3D __wait_for_completion_io_timeout(x, t); \ sdt_might_sleep_end(); \ __ret; \ @@ -195,7 +199,7 @@ extern void complete_all(struct completion *); ({ \ long __ret; \ \ - sdt_might_sleep_start_timeout(NULL, t); \ + sdt_might_sleep_start_timeout((x)->dmap, t); \ __ret =3D __wait_for_completion_interruptible_timeout(x, t); \ sdt_might_sleep_end(); \ __ret; \ @@ -204,7 +208,7 @@ extern void complete_all(struct completion *); ({ \ long __ret; \ \ - sdt_might_sleep_start_timeout(NULL, t); \ + sdt_might_sleep_start_timeout((x)->dmap, t); \ __ret =3D __wait_for_completion_killable_timeout(x, t); \ sdt_might_sleep_end(); \ __ret; \ --=20 2.17.1 From nobody Fri Dec 19 15:51:27 2025 Received: from invmail4.hynix.com (exvmail4.hynix.com [166.125.252.92]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 6BAD9283133; Mon, 19 May 2025 09:19:04 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=166.125.252.92 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747646349; cv=none; b=sXMt2oMB9WDhuIqA/O55dV4Fbd5ihj45pxL7BEQ6pqLn2RGA7rfFa/GoytRBIDoDgxdLvADIHxj3tuvko/reggAYqtWkIc0p0PkGUdxCX57afXHySCwS5UIqYbF5VHcT1TEMoH87MGOLgJhoeqaJsNeDodaPXUJU17DGoqDCy9Q= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747646349; c=relaxed/simple; bh=HDwbv5mCqKj6x3UYAO4JqjIS/q3EH8R7z46aNpP4fec=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References; b=Fx+OjhUCucd03R9PTM6r88CazKT7+hZcuKLQmMohtuLaYjwz1OaerkykrZs/xoE54NGVKDQMUMQMI2GAOzcUVX3KYBCBLUC1LrapQ2ct+CckdGK+dQyCmT31mkj7Y5HLNBRgIdj1iqqV4UPWaK8peXfx5WjDSu3ofiNBh/s7fqY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com; spf=pass smtp.mailfrom=sk.com; arc=none smtp.client-ip=166.125.252.92 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=sk.com X-AuditID: a67dfc5b-669ff7000002311f-26-682af771fa6d From: Byungchul Park To: linux-kernel@vger.kernel.org Cc: kernel_team@skhynix.com, torvalds@linux-foundation.org, damien.lemoal@opensource.wdc.com, linux-ide@vger.kernel.org, adilger.kernel@dilger.ca, linux-ext4@vger.kernel.org, mingo@redhat.com, peterz@infradead.org, will@kernel.org, tglx@linutronix.de, rostedt@goodmis.org, joel@joelfernandes.org, sashal@kernel.org, daniel.vetter@ffwll.ch, duyuyang@gmail.com, johannes.berg@intel.com, tj@kernel.org, tytso@mit.edu, willy@infradead.org, david@fromorbit.com, amir73il@gmail.com, gregkh@linuxfoundation.org, kernel-team@lge.com, linux-mm@kvack.org, akpm@linux-foundation.org, mhocko@kernel.org, minchan@kernel.org, hannes@cmpxchg.org, vdavydov.dev@gmail.com, sj@kernel.org, jglisse@redhat.com, dennis@kernel.org, cl@linux.com, penberg@kernel.org, rientjes@google.com, vbabka@suse.cz, ngupta@vflare.org, linux-block@vger.kernel.org, josef@toxicpanda.com, linux-fsdevel@vger.kernel.org, jack@suse.cz, jlayton@kernel.org, dan.j.williams@intel.com, hch@infradead.org, djwong@kernel.org, dri-devel@lists.freedesktop.org, rodrigosiqueiramelo@gmail.com, melissa.srw@gmail.com, hamohammed.sa@gmail.com, harry.yoo@oracle.com, chris.p.wilson@intel.com, gwan-gyeong.mun@intel.com, max.byungchul.park@gmail.com, boqun.feng@gmail.com, longman@redhat.com, yskelg@gmail.com, yunseong.kim@ericsson.com, yeoreum.yun@arm.com, netdev@vger.kernel.org, matthew.brost@intel.com, her0gyugyu@gmail.com Subject: [PATCH v16 38/42] dept: introduce a new type of dependency tracking between multi event sites Date: Mon, 19 May 2025 18:18:22 +0900 Message-Id: <20250519091826.19752-39-byungchul@sk.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20250519091826.19752-1-byungchul@sk.com> References: <20250519091826.19752-1-byungchul@sk.com> X-Brightmail-Tracker: H4sIAAAAAAAAAzWSa0yTZxTHfd47HSWv1c0HWOJsoigqDqPmmOiybB98vpi47INRErWxL7QR UIui1ZiAXNRy8TKQzAuWqrVpq2KLRoQSLDfBG0pXEAsKMVwiFwO04zrXuvnl5Jf/Oed3vhyB VjSxUYI29ZCkS1UlKzkZIxsOL1t98O9YzY83qiLBP3GagSt37Ry03rEhsFdkUjDYsAXaA0MI Zp6/pKGkuBVBWU8XDRWN3QhclpMctH2IAI9/lIPm4jwOsq7f5eDVx1kKfBcvUGBzbIV35j4G np4zUVAyyMHlkiwqWAYomDJbeTBnLIVeyyUeZnviobnby4KrcyX8WerjoNrVzEDjw14K2h5d 4aDb/pmFp41PGAgURkPr+QIWbo+YOPgYMNNg9o/y8LrWSEGj8Tsozw4Kc8f/YaGpoJaC3Bv3 KPC8qUJQc/o9BQ67l4M6/xAFTkcxDdO3GhD0Fg7zkJM/xcPlzEIEeTkXGcj2rYeZyeDlqxPx kHmtnIHbc17082ZiL7UjUjc0SpNs5xEy7f+LI66AkSEtJkwqL3XxJLumkydGx2HitMSS69WD FCkb87PEYT3DEcfYBZ4Yhj0UGXnxgt/2/U7ZJrWUrE2XdGt+2iPTVL2bow/8oT7aZGthM1DB bwYUJmBxHb7vttBfeeDsPT7EnBiDOzqmvuQLxR+ws6CPNSCZQIveb3D71TfIgARhgZiIiz6F h2YYcSluufWAC7Fc3IBnc/L/dy7GtvLaLxwWzDvz6lCIFeJ67LGVMiEnFkvCsLtoGv23EIkf WzqYc0huRPOsSKFNTU9RaZPXxWn0qdqjcXv3pzhQ8L3MJ2YTHqKx1t/dSBSQMlxe7lqhUbCq 9DR9ihthgVYulFudyzUKuVqlPybp9u/WHU6W0twoWmCUi+RrA0fUCjFJdUjaJ0kHJN3XLiWE RWWgUwlxuYkxaydjKrmarAR3xUD/tldRuOrm9v63Yv3Ekl+KTNcSeyo/bfz2fKRvbv7jtobR vJk46/iGjo319q490ZF69aqt3r7EkR0+vHnTspxfdxhMEen51Z7dn+fnP2tPMoxUVu87Pr7k bG1skmZyy7y6lgESoe/fpSw6tthXb8n9oGTSNKr4WFqXpvoXqpJAwloDAAA= X-Brightmail-Tracker: H4sIAAAAAAAAAzXSf0zMcRgHcJ/v746zr9P4LjPttoZM3Ihnfq35gy8bY/4w/KGbvrpTndwp akzpSi5XQjVcOcWV7krd5WcuKZXyq3SSpGhGN3FG33QU7ph/nr32PM/ezz8Pg8uukEGMWrNf 0GqUsXJKQkg2Lk+bH/89VLXQkLkMxOFMAkxXbRS0V1oR2GpSMXA3rYUXI0MIfj5+ikNBXjuC i29f41DT3IfAWXaUgs53k8EleihozcuiIK3kKgUdH8cw6M0/hYHVvgH6Le8JeHiyGIMCNwXn C9IwXxnEwGspp8GSEgIDZedoGHurgNa+LhIaC1tJcPbMg7NFvRTccbYS0HxzAIPO2yYK+my/ SXjY/ICAkewZ0J5rJKHiczEFH0csOFhEDw3P6s0YNJunQZXel5rx7RcJLcZ6DDIuVWPgelmL oC7zDQZ2WxcFjeIQBg57Hg4/SpsQDGR/oiH9hJeG86nZCLLS8wnQ94bDz1Hf5cJhBaReqCKg YrwLRazibUU2xDcOeXBe7zjA/xCfU7xzxEzwbcUcf+vca5rX1/XQvNmewDvKQvmSO26Mv/hV JHl7+XGKt389RfOGTy6M//zkCb1p5nbJiighVp0oaBesipSoavvH8fjTUQdbrG1kCjJuNqAA hmMXc4M51bTfFDub6+724n4HssGcw/ieNCAJg7NdE7kXhS+RATHMVHY3d+bLJP8OwYZwbaXX Kb+l7BJuLP0E/i9zFmetqv/rAF+/J6sR+S1jwzmXtYg4iSRmNKEcBao1iXFKdWx4mC5GlaRR HwzbtTfOjnwPZDk8lnsTDXeubUAsg+STpFXOuSoZqUzUJcU1II7B5YHScscclUwapUxKFrR7 d2oTYgVdA5rBEPLp0vVbhUgZG63cL8QIQryg/T/FmICgFBTtmm28O+/xVEvJttNhYl5tyJ6I hO6CD9S1Ss94/r34CLeixvQoeM3K71ve3GjJrV6NHcuYnjPU1o7UMrN5q/e5Y8IhafDhKWfj TKN3RXfShuTLHYOuUU0QNJm8g4qehBX995+Sfb+Jjmixd5HkiOfVDv2RR9Hr9u1SJNfkyBcv lRM6lVIRimt1yj8wWfZlPAMAAA== X-CFilter-Loop: Reflected Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" It's worth reporting wait-event circular dependency even if it doesn't lead to an actual deadlock, because it's a good information about a circular dependency anyway. However, it should be suppressed once turning out it doesn't lead an actual deadlock, for instance, there are other wake-up(or event) paths. The report needs to be suppressed by annotating that an event can be recovered by other sites triggering the desired wake-up, using a newly introduced API, dept_recover_event() specifying an event site and its recover site. By the introduction, need of a new type of dependency tracking arises since a loop of recover dependency could trigger another type of deadlock. So implement a logic to track the new type of dependency between multi event sites for a single wait. Lastly, to make sure that recover sites must be used in code, introduce a section '.dept.event_sites' to mark it as 'used' only if used in code, and warn it if dept_recover_event()s are annotated with recover sites, not used in code. Signed-off-by: Byungchul Park --- include/asm-generic/vmlinux.lds.h | 13 +- include/linux/dept.h | 91 ++++++++++++++ kernel/dependency/dept.c | 196 ++++++++++++++++++++++++++++++ 3 files changed, 299 insertions(+), 1 deletion(-) diff --git a/include/asm-generic/vmlinux.lds.h b/include/asm-generic/vmlinu= x.lds.h index 58a635a6d5bd..fd24b35fb379 100644 --- a/include/asm-generic/vmlinux.lds.h +++ b/include/asm-generic/vmlinux.lds.h @@ -699,6 +699,16 @@ defined(CONFIG_AUTOFDO_CLANG) || defined(CONFIG_PROPEL= LER_CLANG) #define KERNEL_CTORS() #endif =20 +#ifdef CONFIG_DEPT +#define DEPT_EVNET_SITES_USED() \ + . =3D ALIGN(8); \ + __dept_event_sites_start =3D .; \ + KEEP(*(.dept.event_sites)) \ + __dept_event_sites_end =3D .; +#else +#define DEPT_EVNET_SITES_USED() +#endif + /* init and exit section handling */ #define INIT_DATA \ KEEP(*(SORT(___kentry+*))) \ @@ -723,7 +733,8 @@ defined(CONFIG_AUTOFDO_CLANG) || defined(CONFIG_PROPELL= ER_CLANG) EARLYCON_TABLE() \ LSM_TABLE() \ EARLY_LSM_TABLE() \ - KUNIT_INIT_TABLE() + KUNIT_INIT_TABLE() \ + DEPT_EVNET_SITES_USED() =20 #define INIT_TEXT \ *(.init.text .init.text.*) \ diff --git a/include/linux/dept.h b/include/linux/dept.h index b164f74e86e5..988aceee36ad 100644 --- a/include/linux/dept.h +++ b/include/linux/dept.h @@ -390,6 +390,82 @@ struct dept_ext_wgen { unsigned int wgen; }; =20 +struct dept_event_site { + /* + * event site name + */ + const char *name; + + /* + * function name where the event is triggered in + */ + const char *func_name; + + /* + * for associating its recover dependencies + */ + struct list_head dep_head; + struct list_head dep_rev_head; + + /* + * for BFS + */ + unsigned int bfs_gen; + struct dept_event_site *bfs_parent; + struct list_head bfs_node; + + /* + * flag indicating the event is not only declared but also + * actually used in code + */ + bool used; +}; + +struct dept_event_site_dep { + struct dept_event_site *evt_site; + struct dept_event_site *recover_site; + + /* + * for linking to dept_event objects + */ + struct list_head dep_node; + struct list_head dep_rev_node; +}; + +#define DEPT_EVENT_SITE_INITIALIZER(es) \ +{ \ + .name =3D #es, \ + .func_name =3D NULL, \ + .dep_head =3D LIST_HEAD_INIT((es).dep_head), \ + .dep_rev_head =3D LIST_HEAD_INIT((es).dep_rev_head), \ + .bfs_gen =3D 0, \ + .bfs_parent =3D NULL, \ + .bfs_node =3D LIST_HEAD_INIT((es).bfs_node), \ + .used =3D false, \ +} + +#define DEPT_EVENT_SITE_DEP_INITIALIZER(esd) \ +{ \ + .evt_site =3D NULL, \ + .recover_site =3D NULL, \ + .dep_node =3D LIST_HEAD_INIT((esd).dep_node), \ + .dep_rev_node =3D LIST_HEAD_INIT((esd).dep_rev_node), \ +} + +struct dept_event_site_init { + struct dept_event_site *evt_site; + const char *func_name; +}; + +#define dept_event_site_used(es) \ +do { \ + static struct dept_event_site_init _evtinit __initdata =3D \ + { .evt_site =3D (es), .func_name =3D __func__ }; \ + static struct dept_event_site_init *_evtinitp __used \ + __attribute__((__section__(".dept.event_sites"))) =3D \ + &_evtinit; \ +} while (0) + extern void dept_stop_emerg(void); extern void dept_on(void); extern void dept_off(void); @@ -427,6 +503,14 @@ static inline void dept_ecxt_enter_nokeep(struct dept_= map *m) extern void dept_key_init(struct dept_key *k); extern void dept_key_destroy(struct dept_key *k); extern void dept_map_ecxt_modify(struct dept_map *m, unsigned long e_f, st= ruct dept_key *new_k, unsigned long new_e_f, unsigned long new_ip, const ch= ar *new_c_fn, const char *new_e_fn, int new_sub_l); +extern void __dept_recover_event(struct dept_event_site_dep *esd, struct d= ept_event_site *es, struct dept_event_site *rs); + +#define dept_recover_event(es, rs) \ +do { \ + static struct dept_event_site_dep _esd =3D DEPT_EVENT_SITE_DEP_INITIALIZE= R(_esd);\ + \ + __dept_recover_event(&_esd, es, rs); \ +} while (0) =20 extern void dept_softirq_enter(void); extern void dept_hardirq_enter(void); @@ -440,8 +524,10 @@ extern void dept_hardirqs_off(void); struct dept_key { }; struct dept_map { }; struct dept_ext_wgen { }; +struct dept_event_site { }; =20 #define DEPT_MAP_INITIALIZER(n, k) { } +#define DEPT_EVENT_SITE_INITIALIZER(es) { } =20 #define dept_stop_emerg() do { } while (0) #define dept_on() do { } while (0) @@ -472,6 +558,7 @@ struct dept_ext_wgen { }; #define dept_key_init(k) do { (void)(k); } while (0) #define dept_key_destroy(k) do { (void)(k); } while (0) #define dept_map_ecxt_modify(m, e_f, n_k, n_e_f, n_ip, n_c_fn, n_e_fn, n_s= l) do { (void)(n_k); (void)(n_c_fn); (void)(n_e_fn); } while (0) +#define dept_recover_event(es, rs) do { } while (0) =20 #define dept_softirq_enter() do { } while (0) #define dept_hardirq_enter() do { } while (0) @@ -482,4 +569,8 @@ struct dept_ext_wgen { }; =20 #define dept_set_lockdep_map(m, lockdep_m) do { } while (0) #endif + +#define DECLARE_DEPT_EVENT_SITE(es) extern struct dept_event_site (es) +#define DEFINE_DEPT_EVENT_SITE(es) struct dept_event_site (es) =3D DEPT_EV= ENT_SITE_INITIALIZER(es) + #endif /* __LINUX_DEPT_H */ diff --git a/kernel/dependency/dept.c b/kernel/dependency/dept.c index b5ba6d939932..e14c17b8e197 100644 --- a/kernel/dependency/dept.c +++ b/kernel/dependency/dept.c @@ -973,6 +973,117 @@ static void bfs(void *root, struct bfs_ops *ops, void= *in, void **out) } } =20 +/* + * Recover dependency between event sites + * =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D + * Even though an event is in a chain of wait-event circular dependency, + * the corresponding wait might be woken up by another site triggering + * the desired event. To reflect that, dept allows to annotate the + * recover relationship between event sites using __dept_recover_event(). + * However, that requires to track a new type of dependency between the + * event sites. + */ + +/* + * Print all events in the circle. + */ +static void print_recover_circle(struct dept_event_site *es) +{ + struct dept_event_site *from =3D es->bfs_parent; + struct dept_event_site *to =3D es; + + dept_outworld_enter(); + + pr_warn("=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D\n"); + pr_warn("DEPT: Circular recover dependency has been detected.\n"); + pr_warn("%s %.*s %s\n", init_utsname()->release, + (int)strcspn(init_utsname()->version, " "), + init_utsname()->version, + print_tainted()); + pr_warn("---------------------------------------------------\n"); + + do { + print_spc(1, "event site(%s@%s)\n", from->name, from->func_name); + print_spc(1, "-> event site(%s@%s)\n", to->name, to->func_name); + to =3D from; + from =3D from->bfs_parent; + + if (to !=3D es) + pr_warn("\n"); + } while (to !=3D es); + + pr_warn("---------------------------------------------------\n"); + pr_warn("information that might be helpful\n"); + pr_warn("---------------------------------------------------\n"); + dump_stack(); + + dept_outworld_exit(); +} + +static void bfs_init_recover(void *node, void *in, void **out) +{ + struct dept_event_site *root =3D (struct dept_event_site *)node; + struct dept_event_site_dep *new =3D (struct dept_event_site_dep *)in; + + root->bfs_gen =3D bfs_gen; + new->recover_site->bfs_parent =3D new->evt_site; +} + +static void bfs_extend_recover(struct list_head *h, void *node) +{ + struct dept_event_site *cur =3D (struct dept_event_site *)node; + struct dept_event_site_dep *esd; + + list_for_each_entry(esd, &cur->dep_head, dep_node) { + struct dept_event_site *next =3D esd->recover_site; + + if (bfs_gen =3D=3D next->bfs_gen) + continue; + next->bfs_parent =3D cur; + next->bfs_gen =3D bfs_gen; + list_add_tail(&next->bfs_node, h); + } +} + +static void *bfs_dequeue_recover(struct list_head *h) +{ + struct dept_event_site *es; + + DEPT_WARN_ON(list_empty(h)); + + es =3D list_first_entry(h, struct dept_event_site, bfs_node); + list_del(&es->bfs_node); + return es; +} + +static enum bfs_ret cb_check_recover_dl(void *node, void *in, void **out) +{ + struct dept_event_site *cur =3D (struct dept_event_site *)node; + struct dept_event_site_dep *new =3D (struct dept_event_site_dep *)in; + + if (cur =3D=3D new->evt_site) { + print_recover_circle(new->recover_site); + return BFS_DONE; + } + + return BFS_CONTINUE; +} + +/* + * NOTE: Must be called with dept_lock held. + */ +static void check_recover_dl_bfs(struct dept_event_site_dep *esd) +{ + struct bfs_ops ops =3D { + .bfs_init =3D bfs_init_recover, + .extend =3D bfs_extend_recover, + .dequeue =3D bfs_dequeue_recover, + .callback =3D cb_check_recover_dl, + }; + + bfs((void *)esd->recover_site, &ops, (void *)esd, NULL); +} + /* * Main operations * =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D @@ -3165,8 +3276,78 @@ static void migrate_per_cpu_pool(void) } } =20 +static bool dept_recover_ready; + +void __dept_recover_event(struct dept_event_site_dep *esd, + struct dept_event_site *es, struct dept_event_site *rs) +{ + struct dept_task *dt =3D dept_task(); + unsigned long flags; + + if (unlikely(!dept_working())) + return; + + if (dt->recursive) + return; + + if (!esd || !es || !rs) { + DEPT_WARN_ONCE("All the parameters should be !NULL.\n"); + return; + } + + /* + * Check locklessly if another already has done it for us. + */ + if (READ_ONCE(esd->evt_site)) + return; + + if (!dept_recover_ready) { + DEPT_WARN("Should be called once dept_recover_ready.\n"); + return; + } + + flags =3D dept_enter(); + if (unlikely(!dept_lock())) + goto exit; + + /* + * Check if another already has done it for us with lock held. + */ + if (esd->evt_site) + goto unlock; + + /* + * Can be used as an indicator of whether this + * __dept_recover_event() has been processed or not as well as + * for storing its associated events. + */ + WRITE_ONCE(esd->evt_site, es); + esd->recover_site =3D rs; + + if (!es->used || !rs->used) { + if (!es->used) + DEPT_INFO("dept_event_site %s has never been used.\n", es->name); + if (!rs->used) + DEPT_INFO("dept_event_site %s has never been used.\n", rs->name); + + DEPT_WARN("Cannot track recover dependency with events that never used.\= n"); + goto unlock; + } + + list_add(&esd->dep_node, &es->dep_head); + list_add(&esd->dep_rev_node, &rs->dep_rev_head); + check_recover_dl_bfs(esd); +unlock: + dept_unlock(); +exit: + dept_exit(flags); +} +EXPORT_SYMBOL_GPL(__dept_recover_event); + #define B2KB(B) ((B) / 1024) =20 +extern char __dept_event_sites_start[], __dept_event_sites_end[]; + /* * Should be called after setup_per_cpu_areas() and before no non-boot * CPUs have been on. @@ -3174,6 +3355,21 @@ static void migrate_per_cpu_pool(void) void __init dept_init(void) { size_t mem_total =3D 0; + struct dept_event_site_init **evtinitpp; + + /* + * dept recover dependency tracking works from now on. + */ + for (evtinitpp =3D (struct dept_event_site_init **)__dept_event_sites_sta= rt; + evtinitpp < (struct dept_event_site_init **)__dept_event_sites_end; + evtinitpp++) { + (*evtinitpp)->evt_site->used =3D true; + (*evtinitpp)->evt_site->func_name =3D (*evtinitpp)->func_name; + pr_info("dept_event %s@%s is initialized.\n", + (*evtinitpp)->evt_site->name, + (*evtinitpp)->evt_site->func_name); + } + dept_recover_ready =3D true; =20 local_irq_disable(); dept_per_cpu_ready =3D 1; --=20 2.17.1 From nobody Fri Dec 19 15:51:27 2025 Received: from invmail4.hynix.com (exvmail4.hynix.com [166.125.252.92]) by smtp.subspace.kernel.org (Postfix) with ESMTP id ABA5B283149; Mon, 19 May 2025 09:19:04 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=166.125.252.92 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747646349; cv=none; b=WLBJLRS8q1vp6mEOt2O4qB/GE2WGmq8HF/zEzyP/aZ3TEQG1tFmLaKVoso7W19cDLu1vjtjsD459UYzC+BKC6rZ+pJyUJpVbjFQ9Q89UfozR/am5D2RpLU+WLELzYw1nWYsEfYb+Fp2gPbSHuo9mGeylj+U2nnEwQi+NtSJTiFU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747646349; c=relaxed/simple; bh=s4zHN9XeEk53MhJYeNB0v+KxCbg7we9x407GQpxdMJ0=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References; b=Km0EBEwaq8YTu+VBfVBeMfmO90twvcPSAl2CdZQze0iT3jvk+VowospfgsF9McvD+vblNjWYJttyx4O3CkFc5NIlXz+u+lwAm8Yjdgs7RRCH0Nxnt2kJMXhuD3EJwTBmrm1/162f5XMl/BuCLni9UjSbwCNuxTQKOBYLI7A431M= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com; spf=pass smtp.mailfrom=sk.com; arc=none smtp.client-ip=166.125.252.92 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=sk.com X-AuditID: a67dfc5b-669ff7000002311f-35-682af77122e1 From: Byungchul Park To: linux-kernel@vger.kernel.org Cc: kernel_team@skhynix.com, torvalds@linux-foundation.org, damien.lemoal@opensource.wdc.com, linux-ide@vger.kernel.org, adilger.kernel@dilger.ca, linux-ext4@vger.kernel.org, mingo@redhat.com, peterz@infradead.org, will@kernel.org, tglx@linutronix.de, rostedt@goodmis.org, joel@joelfernandes.org, sashal@kernel.org, daniel.vetter@ffwll.ch, duyuyang@gmail.com, johannes.berg@intel.com, tj@kernel.org, tytso@mit.edu, willy@infradead.org, david@fromorbit.com, amir73il@gmail.com, gregkh@linuxfoundation.org, kernel-team@lge.com, linux-mm@kvack.org, akpm@linux-foundation.org, mhocko@kernel.org, minchan@kernel.org, hannes@cmpxchg.org, vdavydov.dev@gmail.com, sj@kernel.org, jglisse@redhat.com, dennis@kernel.org, cl@linux.com, penberg@kernel.org, rientjes@google.com, vbabka@suse.cz, ngupta@vflare.org, linux-block@vger.kernel.org, josef@toxicpanda.com, linux-fsdevel@vger.kernel.org, jack@suse.cz, jlayton@kernel.org, dan.j.williams@intel.com, hch@infradead.org, djwong@kernel.org, dri-devel@lists.freedesktop.org, rodrigosiqueiramelo@gmail.com, melissa.srw@gmail.com, hamohammed.sa@gmail.com, harry.yoo@oracle.com, chris.p.wilson@intel.com, gwan-gyeong.mun@intel.com, max.byungchul.park@gmail.com, boqun.feng@gmail.com, longman@redhat.com, yskelg@gmail.com, yunseong.kim@ericsson.com, yeoreum.yun@arm.com, netdev@vger.kernel.org, matthew.brost@intel.com, her0gyugyu@gmail.com Subject: [PATCH v16 39/42] dept: add module support for struct dept_event_site and dept_event_site_dep Date: Mon, 19 May 2025 18:18:23 +0900 Message-Id: <20250519091826.19752-40-byungchul@sk.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20250519091826.19752-1-byungchul@sk.com> References: <20250519091826.19752-1-byungchul@sk.com> X-Brightmail-Tracker: H4sIAAAAAAAAAzWSfUzMcRzH+/6eu5z9HPLzsOisIYoM+8ww/MHvn9TmLw9bjn7cTV25S8rY imqndKrtavR0XZx2d+ncMUnXUiudhqOTnoRG03qauCMid5l/Pnvt89779fnnw+CSDnIZo1Cm CCqlLEFKiQjR+DxDxJnv4fJN2voQ8HzTEFBWZ6HAdceMwHIvE4ORtv3wxjuG4NezFziU6FwI qj68xeFe+yACR80lCro+zge3Z5ICpy6PgsvVdRS8HJ3BYKC4CAOzLRreGYcJ6CwwYFAyQkFp yWXMNz5jMG000WDMCIOhmhs0zHyIAudgNwmOvvVwvWKAgkaHk4D2+iEMuhrKKBi0zJLQ2d5B gFe7HFyF+STUThgoGPUacTB6Jml41azHoF0fDNYsnzDn6x8SnuQ3Y5Bz8y4G7t5HCJo07zGw WbopaPWMYWC36XD4ebsNwZB2nIbsq9M0lGZqEeRlFxOQNbAVfv3wXS7/FgWZlVYCan93o907 eUuFBfGtY5M4n2U/x//0vKZ4h1dP8E8NHP/wxluaz2rqo3m97SxvrwnnqxtHML5qykPyNtMV irdNFdF87rgb4yeeP6djVxwW7YgXEhSpgmrjrmMieXFBJp08ui8tzzpOZqCb23NRIMOxW7gH Hiv9n29V5swxxa7henqmcT8vYldx9vxhMheJGJztDuLelPcif7CQPckV9/aQfibYMM5cbqT8 LGa3cROFXeQ/6UrObG2eEwX69n15rXNdCbuVc5srCL+UY0sCuY7+BvxfYSn3uKaHKEBiPQow IYlCmZooUyRsiZSnKxVpkSeSEm3I91/GizNH6tGU62ALYhkknSe2OtbJJaQsVZ2e2II4Bpcu Epvsa+UScbws/bygSopTnU0Q1C1oOUNIl4g3e8/FS9hTshThtCAkC6r/KcYELstAbOr54L17 I5T9ufdnzQFfNJp+531JuDc6SBsLzUMLTbTbXXGpLlscs+tCCHnEHU1uNywIenqSK310aH/Y J9uea8dD4wqN0wePPxTdrRxbExK9+XuoLqmUVrZdPzq8mgqb3aA+UVSoU3zRMCtdp4KdVWnV tzs+dc4sbjiQsiOSjJESarksKhxXqWV/AVdh1idbAwAA X-Brightmail-Tracker: H4sIAAAAAAAAAzWSe0hTcRTH/d3nXC1uU+qi9GBkDytL0jhUVBTUrdCif4IicuTNjXTalqZC 4HJZaYpWzrKc02rJNmtu/tFrZk6ttTJLMydqKmLJdEI5y7LHVvTP4cP5cj7n/HEEuNhIhgnk ipO8UiFNllBCQhi/MXf1ia+RsrU1Qxj4Js8TcOOemYL2uyYE5no1BqMtO+H91BiCH69e41BW 2o6garAPh/rWfgT2mjMUdAzPgU7fBAXO0gIKcm/eo+CNZwaDXu0lDEzWOPhgGCHAVVyNQdko BdfLcjF/+YTBtMFIgyEnAoZqymmYGYwGZ38XCY4KJwn2npVwTddLwWO7k4DW+/7rOh7eoKDf /JsEV+tzAqaKwqG9pJCEWm81BZ4pAw4G3wQNbxv1GLTq54FF47fmfflFwrPCRgzybtVh0Ol+ hKDh/AAGVnMXBQ7fGAY2aykO3++0IBgqGqfh7MVpGq6rixAUnNUSoOmNhR/f/JsrJqNBXWkh oPZnF9q6mTPrzIhzjE3gnMZ2ivvue0dx9ik9wb2oZrkH5X00p2nooTm9NZ2z1URyNx+PYlzV Zx/JWY0XKM76+RLN5Y93Ypy3rY3et+CgcFMinyzP4JVrNicIZdpiNZ3m2ZFZYBknc9CtDfko WMAyMeztyjw6wBSzjO3unsYDHMosZm2FI2Q+EgpwpmsW+77CjQJBCHOM1bq7yQATTARrqjBQ ARYx61lvSQf5T7qINVka/4qC/f2eAsffWTETy3aadEQxEupRkBGFyhUZKVJ5cmyU6rgsSyHP jDqammJF/g8ynJ4puY8mO3Y2IUaAJLNFFvsKmZiUZqiyUpoQK8AloSKjbblMLEqUZmXzytQj yvRkXtWEwgWEZL5o9wE+QcwkSU/yx3k+jVf+TzFBcFgOGmhhqlzuoDh3+tJz0iVpo0vcQoc3 cwCXzLwMcxw6nKTJIbbtbZ5OvFKcHT53ck6VuztkVo8raW555NiWp6Euj9oL5+Qb18Y8KVlW u0pxeSBiXZDz68LElXd046kn4vtG2i7v9a5r3s8GPcjNsGq9CR+VB66ermvf7ojdM6zbleKR ECqZNDoSV6qkfwC8Kk4PPQMAAA== X-CFilter-Loop: Reflected Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" struct dept_event_site and struct dept_event_site_dep have been introduced to track dependencies between multi event sites for a single wait, that will be loaded to data segment. Plus, a custom section, '.dept.event_sites', also has been introduced to keep pointers to the objects to make sure all the event sites defined exist in code. dept should work with the section and segment of module. Add the support to handle the section and segment properly whenever modules are loaded and unloaded. Signed-off-by: Byungchul Park --- include/linux/dept.h | 14 +++++++ include/linux/module.h | 5 +++ kernel/dependency/dept.c | 79 +++++++++++++++++++++++++++++++++++----- kernel/module/main.c | 15 ++++++++ 4 files changed, 103 insertions(+), 10 deletions(-) diff --git a/include/linux/dept.h b/include/linux/dept.h index 988aceee36ad..25fdd324614a 100644 --- a/include/linux/dept.h +++ b/include/linux/dept.h @@ -414,6 +414,11 @@ struct dept_event_site { struct dept_event_site *bfs_parent; struct list_head bfs_node; =20 + /* + * for linking all dept_event_site's + */ + struct list_head all_node; + /* * flag indicating the event is not only declared but also * actually used in code @@ -430,6 +435,11 @@ struct dept_event_site_dep { */ struct list_head dep_node; struct list_head dep_rev_node; + + /* + * for linking all dept_event_site_dep's + */ + struct list_head all_node; }; =20 #define DEPT_EVENT_SITE_INITIALIZER(es) \ @@ -441,6 +451,7 @@ struct dept_event_site_dep { .bfs_gen =3D 0, \ .bfs_parent =3D NULL, \ .bfs_node =3D LIST_HEAD_INIT((es).bfs_node), \ + .all_node =3D LIST_HEAD_INIT((es).all_node), \ .used =3D false, \ } =20 @@ -450,6 +461,7 @@ struct dept_event_site_dep { .recover_site =3D NULL, \ .dep_node =3D LIST_HEAD_INIT((esd).dep_node), \ .dep_rev_node =3D LIST_HEAD_INIT((esd).dep_rev_node), \ + .all_node =3D LIST_HEAD_INIT((esd).all_node), \ } =20 struct dept_event_site_init { @@ -473,6 +485,7 @@ extern void dept_init(void); extern void dept_task_init(struct task_struct *t); extern void dept_task_exit(struct task_struct *t); extern void dept_free_range(void *start, unsigned int sz); +extern void dept_mark_event_site_used(void *start, void *end); =20 extern void dept_map_init(struct dept_map *m, struct dept_key *k, int sub_= u, const char *n); extern void dept_map_reinit(struct dept_map *m, struct dept_key *k, int su= b_u, const char *n); @@ -536,6 +549,7 @@ struct dept_event_site { }; #define dept_task_init(t) do { } while (0) #define dept_task_exit(t) do { } while (0) #define dept_free_range(s, sz) do { } while (0) +#define dept_mark_event_site_used(s, e) do { } while (0) =20 #define dept_map_init(m, k, su, n) do { (void)(n); (void)(k); } while (0) #define dept_map_reinit(m, k, su, n) do { (void)(n); (void)(k); } while = (0) diff --git a/include/linux/module.h b/include/linux/module.h index b3329110d668..3da466a2c705 100644 --- a/include/linux/module.h +++ b/include/linux/module.h @@ -28,6 +28,7 @@ #include #include #include +#include =20 #include #include @@ -599,6 +600,10 @@ struct module { #ifdef CONFIG_DYNAMIC_DEBUG_CORE struct _ddebug_info dyndbg_info; #endif +#ifdef CONFIG_DEPT + struct dept_event_site **dept_event_sites; + unsigned int num_dept_event_sites; +#endif } ____cacheline_aligned __randomize_layout; #ifndef MODULE_ARCH_INIT #define MODULE_ARCH_INIT {} diff --git a/kernel/dependency/dept.c b/kernel/dependency/dept.c index e14c17b8e197..baa60bd0fb93 100644 --- a/kernel/dependency/dept.c +++ b/kernel/dependency/dept.c @@ -984,6 +984,9 @@ static void bfs(void *root, struct bfs_ops *ops, void *= in, void **out) * event sites. */ =20 +static LIST_HEAD(dept_event_sites); +static LIST_HEAD(dept_event_site_deps); + /* * Print all events in the circle. */ @@ -2043,6 +2046,33 @@ static void del_dep_rcu(struct rcu_head *rh) preempt_enable(); } =20 +/* + * NOTE: Must be called with dept_lock held. + */ +static void disconnect_event_site_dep(struct dept_event_site_dep *esd) +{ + list_del_rcu(&esd->dep_node); + list_del_rcu(&esd->dep_rev_node); +} + +/* + * NOTE: Must be called with dept_lock held. + */ +static void disconnect_event_site(struct dept_event_site *es) +{ + struct dept_event_site_dep *esd, *next_esd; + + list_for_each_entry_safe(esd, next_esd, &es->dep_head, dep_node) { + list_del_rcu(&esd->dep_node); + list_del_rcu(&esd->dep_rev_node); + } + + list_for_each_entry_safe(esd, next_esd, &es->dep_rev_head, dep_rev_node) { + list_del_rcu(&esd->dep_node); + list_del_rcu(&esd->dep_rev_node); + } +} + /* * NOTE: Must be called with dept_lock held. */ @@ -2384,6 +2414,8 @@ void dept_free_range(void *start, unsigned int sz) { struct dept_task *dt =3D dept_task(); struct dept_class *c, *n; + struct dept_event_site_dep *esd, *next_esd; + struct dept_event_site *es, *next_es; unsigned long flags; =20 if (unlikely(!dept_working())) @@ -2405,6 +2437,24 @@ void dept_free_range(void *start, unsigned int sz) while (unlikely(!dept_lock())) cpu_relax(); =20 + list_for_each_entry_safe(esd, next_esd, &dept_event_site_deps, all_node) { + if (!within((void *)esd, start, sz)) + continue; + + disconnect_event_site_dep(esd); + list_del(&esd->all_node); + } + + list_for_each_entry_safe(es, next_es, &dept_event_sites, all_node) { + if (!within((void *)es, start, sz) && + !within(es->name, start, sz) && + !within(es->func_name, start, sz)) + continue; + + disconnect_event_site(es); + list_del(&es->all_node); + } + list_for_each_entry_safe(c, n, &dept_classes, all_node) { if (!within((void *)c->key, start, sz) && !within(c->name, start, sz)) @@ -3336,6 +3386,7 @@ void __dept_recover_event(struct dept_event_site_dep = *esd, =20 list_add(&esd->dep_node, &es->dep_head); list_add(&esd->dep_rev_node, &rs->dep_rev_head); + list_add(&esd->all_node, &dept_event_site_deps); check_recover_dl_bfs(esd); unlock: dept_unlock(); @@ -3346,6 +3397,23 @@ EXPORT_SYMBOL_GPL(__dept_recover_event); =20 #define B2KB(B) ((B) / 1024) =20 +void dept_mark_event_site_used(void *start, void *end) +{ + struct dept_event_site_init **evtinitpp; + + for (evtinitpp =3D (struct dept_event_site_init **)start; + evtinitpp < (struct dept_event_site_init **)end; + evtinitpp++) { + (*evtinitpp)->evt_site->used =3D true; + (*evtinitpp)->evt_site->func_name =3D (*evtinitpp)->func_name; + list_add(&(*evtinitpp)->evt_site->all_node, &dept_event_sites); + + pr_info("dept_event_site %s@%s is initialized.\n", + (*evtinitpp)->evt_site->name, + (*evtinitpp)->evt_site->func_name); + } +} + extern char __dept_event_sites_start[], __dept_event_sites_end[]; =20 /* @@ -3355,20 +3423,11 @@ extern char __dept_event_sites_start[], __dept_even= t_sites_end[]; void __init dept_init(void) { size_t mem_total =3D 0; - struct dept_event_site_init **evtinitpp; =20 /* * dept recover dependency tracking works from now on. */ - for (evtinitpp =3D (struct dept_event_site_init **)__dept_event_sites_sta= rt; - evtinitpp < (struct dept_event_site_init **)__dept_event_sites_end; - evtinitpp++) { - (*evtinitpp)->evt_site->used =3D true; - (*evtinitpp)->evt_site->func_name =3D (*evtinitpp)->func_name; - pr_info("dept_event %s@%s is initialized.\n", - (*evtinitpp)->evt_site->name, - (*evtinitpp)->evt_site->func_name); - } + dept_mark_event_site_used(__dept_event_sites_start, __dept_event_sites_en= d); dept_recover_ready =3D true; =20 local_irq_disable(); diff --git a/kernel/module/main.c b/kernel/module/main.c index 7e569e1b4db5..1c439ed8c9ed 100644 --- a/kernel/module/main.c +++ b/kernel/module/main.c @@ -2611,6 +2611,11 @@ static int find_module_sections(struct module *mod, = struct load_info *info) &mod->dyndbg_info.num_classes); #endif =20 +#ifdef CONFIG_DEPT + mod->dept_event_sites =3D section_objs(info, ".dept.event_sites", + sizeof(*mod->dept_event_sites), + &mod->num_dept_event_sites); +#endif return 0; } =20 @@ -3249,6 +3254,14 @@ static int early_mod_check(struct load_info *info, i= nt flags) return err; } =20 +static void dept_mark_event_site_used_module(struct module *mod) +{ +#ifdef CONFIG_DEPT + dept_mark_event_site_used(mod->dept_event_sites, + mod->dept_event_sites + mod->num_dept_event_sites); +#endif +} + /* * Allocate and load the module: note that size of section 0 is always * zero, and we rely on this for optional sections. @@ -3408,6 +3421,8 @@ static int load_module(struct load_info *info, const = char __user *uargs, /* Done! */ trace_module_load(mod); =20 + dept_mark_event_site_used_module(mod); + return do_init_module(mod); =20 sysfs_cleanup: --=20 2.17.1 From nobody Fri Dec 19 15:51:27 2025 Received: from invmail4.hynix.com (exvmail4.hynix.com [166.125.252.92]) by smtp.subspace.kernel.org (Postfix) with ESMTP id C009D28315F; Mon, 19 May 2025 09:19:04 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=166.125.252.92 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747646348; cv=none; b=r8YpL9YR+bWVxBNHeBqHFloTLDcp/9BPd3Ga6WM+wbS/NxphHbqa3AUy1C729pfzfCl5s/qIBkw36vNwvm+gE2C/0UqumBGAaAdDtvF7OYIxsV5+lK6M4nYSqm1CV2IcrLv0SZpu0+RoEJXNDbW9AUIjZYcVufuJ0Ttpf3XFlY4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747646348; c=relaxed/simple; bh=5iNxUiZ0D3cUVfQSKToDeMBtYk+ALyH8lFAmQxiYpMs=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References; b=MC6sz3cfGOKeYHG963XWAW4hiu6NZUlDr8YGInvx3U1ev+n+++dYSraTlTjM/G3wRp+ov+J9kbDK+/BN9jAXF0vJcOWKtwmn9sgvoY5OUD9PYckFpKtwdn9/059OW0trwwlA/Pt38NMC0ON3jK4iJk58FzDZ2YOHy/wZlwoOuC8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com; spf=pass smtp.mailfrom=sk.com; arc=none smtp.client-ip=166.125.252.92 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=sk.com X-AuditID: a67dfc5b-669ff7000002311f-44-682af771e118 From: Byungchul Park To: linux-kernel@vger.kernel.org Cc: kernel_team@skhynix.com, torvalds@linux-foundation.org, damien.lemoal@opensource.wdc.com, linux-ide@vger.kernel.org, adilger.kernel@dilger.ca, linux-ext4@vger.kernel.org, mingo@redhat.com, peterz@infradead.org, will@kernel.org, tglx@linutronix.de, rostedt@goodmis.org, joel@joelfernandes.org, sashal@kernel.org, daniel.vetter@ffwll.ch, duyuyang@gmail.com, johannes.berg@intel.com, tj@kernel.org, tytso@mit.edu, willy@infradead.org, david@fromorbit.com, amir73il@gmail.com, gregkh@linuxfoundation.org, kernel-team@lge.com, linux-mm@kvack.org, akpm@linux-foundation.org, mhocko@kernel.org, minchan@kernel.org, hannes@cmpxchg.org, vdavydov.dev@gmail.com, sj@kernel.org, jglisse@redhat.com, dennis@kernel.org, cl@linux.com, penberg@kernel.org, rientjes@google.com, vbabka@suse.cz, ngupta@vflare.org, linux-block@vger.kernel.org, josef@toxicpanda.com, linux-fsdevel@vger.kernel.org, jack@suse.cz, jlayton@kernel.org, dan.j.williams@intel.com, hch@infradead.org, djwong@kernel.org, dri-devel@lists.freedesktop.org, rodrigosiqueiramelo@gmail.com, melissa.srw@gmail.com, hamohammed.sa@gmail.com, harry.yoo@oracle.com, chris.p.wilson@intel.com, gwan-gyeong.mun@intel.com, max.byungchul.park@gmail.com, boqun.feng@gmail.com, longman@redhat.com, yskelg@gmail.com, yunseong.kim@ericsson.com, yeoreum.yun@arm.com, netdev@vger.kernel.org, matthew.brost@intel.com, her0gyugyu@gmail.com Subject: [PATCH v16 40/42] dept: introduce event_site() to disable event tracking if it's recoverable Date: Mon, 19 May 2025 18:18:24 +0900 Message-Id: <20250519091826.19752-41-byungchul@sk.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20250519091826.19752-1-byungchul@sk.com> References: <20250519091826.19752-1-byungchul@sk.com> X-Brightmail-Tracker: H4sIAAAAAAAAAzXSbUwTdxgA8P3/d727FuuOyvB8yWaaGJVFBaPm+SAGEsVbsg0TkyXqFneR 29oIRVtEmJoAIkE6iJpApwi2vNSmLVpbM0EoshIK6OZAWXkZolRlIiAGaRFEXYH45ckvz+uX hyEUrZLljFqTJmo1QrKSkpGysUWm9UemolTRN++vg8BkPgmXrtkp6LhqQ2C/kY1huGUXdAdH Ebz9628CDMUdCEyDDwm44R1A4LbkUPDg6WLoCoxT0F6sp+BU5TUKOkdmMfSXnMdgc34Dj8xD JNw9W4HBMExBqeEUDoXnGKbNVhrMWavBb7lIw+xgDLQP+CTg7vsSLpT3U9DgbifBW+vH8ODW JQoG7B8kcNfbRkKwaAV0nCuUQM3LCgpGgmYCzIFxGu43GTF4jZHgyA0tzHv9XgKthU0Y8qqu Y+jqrUfQmP8Yg9Puo6A5MIrB5SwmYOZKCwJ/0RgNp3+dpqE0uwiB/nQJCbn9W+Dtm9DlsskY yL7sIKHmnQ/FxfL2cjvim0fHCT7XdYyfCfxD8e6gkeTvVHB83cWHNJ/b2EfzRudR3mWJ4isb hjFvmghIeKf1DMU7J87TfMFYF+Zf3rtH7165T7YtSUxWp4vajdt/lKn+PVeFDj/5PCMnrwZn oWquAEkZjt3MDZnq0EfnzDyeN8Wu4Xp6pok5R7CrOFfhkKQAyRiC9YVx3WW9801LWJFz+Kbw nEl2Nfeu/hk1Zzm7lfNXdhILS7/gbI6meUtD+T598/ysgt3CddnKyYUeg5Trm/x+wcu4Pyw9 5FkkN6JPrEih1qSnCOrkzRtUmRp1xoaDqSlOFHov88nZ/bVoomOPB7EMUi6SO9zrVAqJkK7L TPEgjiGUEXKra61KIU8SMn8RtakHtEeTRZ0HrWBI5VL5puCxJAX7s5AmHhLFw6L2YxUz0uVZ aFl4+d5Dv8d9NvKG/DQorR6s7pY6diT0xvu8utLiaE/S17u3e76a+lDLGO7Qi18FI4deTIYn 7EjbJv3ptuVZzprO1CNl3/5ZcuKHpfq242HxpWZbQoU/MT/juyyb/bdG6wl9XFjsdWtllcEk 1Mb/t5FdWRep6bwQmyrs2+mjNzUn5ilJnUqIiSK0OuF/EerFUloDAAA= X-Brightmail-Tracker: H4sIAAAAAAAAAzWSf0zMcRjHfT7fX3eXs+9O4yvz68ZUfrbJHmOG2XwxstnY/KOj79xRp+4q YqjrtPSLY2n0w1Wcdnfp3N0fyFkr4kSicymVOkQTbXSpU7gy/zx77f1+nvf7n0dEyMxUmEil ThY0akW8nJaQkp1rM5cl/YxUruzuCgH/UDYJJTVWGlpuWRBYnRkY+h9tgbbhAQS/nr8goKiw BUF5bxcBzsZuBK4qHQ2tH6aBxz9Ig7swl4bMyhoaXn4Zw9B5+SIGi30HvDP1kdB0oQJDUT8N xUWZODg+Yxg1mRkwpS8CX9VVBsZ6o8Dd7aWgodRNgatjCVwp66ThvstNQuMdH4bWeyU0dFv/ UNDU+ISE4YLZ0GLIp6D6WwUNX4ZNBJj8gwy8qjNiaDTOAJs+mJr14zcFj/PrMGRdv43B016L 4EF2Dwa71UtDg38Ag8NeSEDg5iMEvoKvDJzNG2WgOKMAQe7ZyyToO6Ph10iwuXQoCjKu2Uio HveiDet5a5kV8Q0DgwSvdxzjA/7XNO8aNpL80wqOv3u1i+H1DzoY3mhP4R1VkXzl/X7Ml3/3 U7zdfI7m7d8vMnzOVw/mvzU3M7vm7JOsixPiVamCZsX6WInyreE6Snw/97guqxqnoxtcDhKL OHYVpwv0oAmm2cXcmzejxASHsvM5R34flYMkIoL1hnBtpe2TS9NZgbN5f+IJJtlF3HjtR3qC pexqzlf5kvgXOo+z2OomWRzUO3IbJm9lbDTnsZSRF5DEiKaYUahKnZqgUMVHL9ceUaapVceX HzyaYEfBDzKdGjPcQUOtW+oRK0LyqVKbK0IpoxSp2rSEesSJCHmo1OwIV8qkcYq0E4Lm6H5N SrygrUezRaR8pnTbXiFWxh5SJAtHBCFR0Px3sUgclo4SLb3TH37a7FmQadg7smNj8Xnxk/DT csq/oscXeJon3fWs79S6kllD+wf3OJ0rI0JeoXnGZ4bx5L6Tn03O+pEzsWJ3ufSSLqZ57r3A uTVhMYfb1oRv2rTQk1g7NvXK5tzftqW4IE5v+dO227wx+UDe4qRbj29uT8reoF6oyzJund+0 VU5qlYqoSEKjVfwFKydO1T0DAAA= X-CFilter-Loop: Reflected Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" With multi event sites for a single wait, dept allows to skip tracking an event that is recoverable by other recover paths. Introduce an API, event_site(), to skip tracking the event in the case. Signed-off-by: Byungchul Park --- include/linux/dept.h | 30 ++++++++++++++++++++++++++++++ include/linux/sched.h | 6 ++++++ kernel/dependency/dept.c | 20 ++++++++++++++++++++ 3 files changed, 56 insertions(+) diff --git a/include/linux/dept.h b/include/linux/dept.h index 25fdd324614a..0ac13129f308 100644 --- a/include/linux/dept.h +++ b/include/linux/dept.h @@ -487,6 +487,31 @@ extern void dept_task_exit(struct task_struct *t); extern void dept_free_range(void *start, unsigned int sz); extern void dept_mark_event_site_used(void *start, void *end); =20 +extern void disable_event_track(void); +extern void enable_event_track(void); + +#define event_site(es, evt_func, ...) \ +do { \ + unsigned long _flags; \ + bool _disable; \ + \ + local_irq_save(_flags); \ + dept_event_site_used(es); \ + /* \ + * If !list_empty(&(es)->dept_head), the event site can be \ + * recovered by others. Do not track event dependency if so. \ + */ \ + _disable =3D !list_empty(&(es)->dep_head); \ + if (_disable) \ + disable_event_track(); \ + \ + evt_func(__VA_ARGS__); \ + \ + if (_disable) \ + enable_event_track(); \ + local_irq_restore(_flags); \ +} while (0) + extern void dept_map_init(struct dept_map *m, struct dept_key *k, int sub_= u, const char *n); extern void dept_map_reinit(struct dept_map *m, struct dept_key *k, int su= b_u, const char *n); extern void dept_ext_wgen_init(struct dept_ext_wgen *ewg); @@ -550,6 +575,11 @@ struct dept_event_site { }; #define dept_task_exit(t) do { } while (0) #define dept_free_range(s, sz) do { } while (0) #define dept_mark_event_site_used(s, e) do { } while (0) +#define event_site(es, evt_func, ...) \ +do { \ + (void)(es); \ + evt_func(__VA_ARGS__); \ +} while (0) =20 #define dept_map_init(m, k, su, n) do { (void)(n); (void)(k); } while (0) #define dept_map_reinit(m, k, su, n) do { (void)(n); (void)(k); } while = (0) diff --git a/include/linux/sched.h b/include/linux/sched.h index 43927e61921b..44a77b7116b7 100644 --- a/include/linux/sched.h +++ b/include/linux/sched.h @@ -873,6 +873,11 @@ struct dept_task { */ int missing_ecxt; =20 + /* + * not to track events + */ + int disable_event_track_cnt; + /* * for tracking IRQ-enable state */ @@ -910,6 +915,7 @@ struct dept_task { .stage_wait_stack =3D NULL, \ .stage_lock =3D (arch_spinlock_t)__ARCH_SPIN_LOCK_UNLOCKED,\ .missing_ecxt =3D 0, \ + .disable_event_track_cnt =3D 0, \ .hardirqs_enabled =3D false, \ .softirqs_enabled =3D false, \ .task_exit =3D false, \ diff --git a/kernel/dependency/dept.c b/kernel/dependency/dept.c index baa60bd0fb93..c65bb0c6dad2 100644 --- a/kernel/dependency/dept.c +++ b/kernel/dependency/dept.c @@ -2573,6 +2573,23 @@ static void __dept_wait(struct dept_map *m, unsigned= long w_f, } } =20 +void disable_event_track(void) +{ + dept_task()->disable_event_track_cnt++; +} +EXPORT_SYMBOL_GPL(disable_event_track); + +void enable_event_track(void) +{ + dept_task()->disable_event_track_cnt--; +} +EXPORT_SYMBOL_GPL(enable_event_track); + +static bool event_track_disabled(void) +{ + return !!dept_task()->disable_event_track_cnt; +} + /* * Called between dept_enter() and dept_exit(). */ @@ -2585,6 +2602,9 @@ static void __dept_event(struct dept_map *m, struct d= ept_map *real_m, struct dept_key *k; int e; =20 + if (event_track_disabled()) + return; + e =3D find_first_bit(&e_f, DEPT_MAX_SUBCLASSES_EVT); =20 if (DEPT_WARN_ON(e >=3D DEPT_MAX_SUBCLASSES_EVT)) --=20 2.17.1 From nobody Fri Dec 19 15:51:27 2025 Received: from invmail4.hynix.com (exvmail4.hynix.com [166.125.252.92]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 620F1283FE4; Mon, 19 May 2025 09:19:05 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=166.125.252.92 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747646350; cv=none; b=EChS7saL2CIsQKEQRZ4sYir1gpyrNO//4SQlBmYp3VBk8TJnHrhr3UqbUW5KVm753+EP/HHJKVvqWLz84NeHoW49AsfxQKF4uo4xP9m4pg7n3z+eZvRLw9Ueb29N3YoYTQWrojzQ/RBKaWvfMH7Nb7DM1oHl69NE2CPGRh6q0Fg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747646350; c=relaxed/simple; bh=4S0kmmxNZl18R3dkj2pAd/m/S3hp3ORwuEcWWn/GYOE=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References; b=qynPph77LAml1ftCVTCEcgdGLPN8KJIkeUd8Q07oN2XI0RC3j2hLiFfWKMkA0SEps6XULKZoqGu4azTNQSbIjRks3G6j53t7zmmoHGGnD5DzGkfA2MCCyUcLbyOLGzBCua6S/M6fC2ZjXPyo1uXs9p7ivDiZR3euJWxm3ia5U0Q= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com; spf=pass smtp.mailfrom=sk.com; arc=none smtp.client-ip=166.125.252.92 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=sk.com X-AuditID: a67dfc5b-669ff7000002311f-54-682af77101aa From: Byungchul Park To: linux-kernel@vger.kernel.org Cc: kernel_team@skhynix.com, torvalds@linux-foundation.org, damien.lemoal@opensource.wdc.com, linux-ide@vger.kernel.org, adilger.kernel@dilger.ca, linux-ext4@vger.kernel.org, mingo@redhat.com, peterz@infradead.org, will@kernel.org, tglx@linutronix.de, rostedt@goodmis.org, joel@joelfernandes.org, sashal@kernel.org, daniel.vetter@ffwll.ch, duyuyang@gmail.com, johannes.berg@intel.com, tj@kernel.org, tytso@mit.edu, willy@infradead.org, david@fromorbit.com, amir73il@gmail.com, gregkh@linuxfoundation.org, kernel-team@lge.com, linux-mm@kvack.org, akpm@linux-foundation.org, mhocko@kernel.org, minchan@kernel.org, hannes@cmpxchg.org, vdavydov.dev@gmail.com, sj@kernel.org, jglisse@redhat.com, dennis@kernel.org, cl@linux.com, penberg@kernel.org, rientjes@google.com, vbabka@suse.cz, ngupta@vflare.org, linux-block@vger.kernel.org, josef@toxicpanda.com, linux-fsdevel@vger.kernel.org, jack@suse.cz, jlayton@kernel.org, dan.j.williams@intel.com, hch@infradead.org, djwong@kernel.org, dri-devel@lists.freedesktop.org, rodrigosiqueiramelo@gmail.com, melissa.srw@gmail.com, hamohammed.sa@gmail.com, harry.yoo@oracle.com, chris.p.wilson@intel.com, gwan-gyeong.mun@intel.com, max.byungchul.park@gmail.com, boqun.feng@gmail.com, longman@redhat.com, yskelg@gmail.com, yunseong.kim@ericsson.com, yeoreum.yun@arm.com, netdev@vger.kernel.org, matthew.brost@intel.com, her0gyugyu@gmail.com Subject: [PATCH v16 41/42] dept: implement a basic unit test for dept Date: Mon, 19 May 2025 18:18:25 +0900 Message-Id: <20250519091826.19752-42-byungchul@sk.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20250519091826.19752-1-byungchul@sk.com> References: <20250519091826.19752-1-byungchul@sk.com> X-Brightmail-Tracker: H4sIAAAAAAAAAzWSeUiTcRzG+/3e09XqZV1vGRWDLrs0Or6d2P0WCEFEdNFWvrShLptpGQme YbMNs3QdZstirbnK3vVHpgtzzLLIrGwe6TKpSLzI2sqyYxr98+XheXg+338ellA8oiayWt1h Ua9TxyppGSnrHlE899C3ME142ePZ4P+aTULhbQcNdbdKEDjupmHo8GyEhkAXgp/PnhNgzq9D cOVdKwF3q30IXLZ0Gl69Hwn1/l4aavJzaMi4epuGF50DGFoK8jCUSFHw1vqRhKe5xRjMHTRc NGfg4PmEod9qZ8CaOg3abRcYGHgXATU+LwWu5tlwvqiFhgpXDQnV99oxvLpfSIPP8YeCp9WP SQiYQqHutJGCmz3FNHQGrARY/b0MvKy0YKi2jIPSzCDwxJffFDwyVmI4ce0OhvqmcgQPstsw SA4vDW5/FwanlE/Aj+seBO2mbgayTvUzcDHNhCAnq4CEzJZF8PN78POlrxGQdrmUhJu/vChy peAociDB3dVLCJnOI8IP/2tacAUspPCkmBfKLrQyQuaDZkawSImC0xYmXK3owMKVPj8lSPaT tCD15TGCobseCz21tcyWSTtlK6LFWG2SqJ+/SiXTGNoycPzZjUcDTiOVijzLDSiE5bmFfLnp DPFff//iGNI0N4NvbOwf0mO4qbzT+JEyIBlLcN7hfMOlJjQYjObW81JZA2NALEty0/g2++pB W84t5m9531D/mFP4ktLKIU5I0G/OcQ9VFdwivr6kiBxk8pw5hDe1euh/hQn8Q1sjmYvkFjTM jhRaXVKcWhu7cJ4mWac9Om//wTgJBedlTRnYdQ/11W2tQhyLlCPkpa5ZGgWlTkpIjqtCPEso x8jtzpkahTxanXxM1B/cq0+MFROqUChLKsfLFwSORCu4A+rDYowoxov6/ylmQyamoqzVk9fE fJiQXTCqJzG30Bd56Np2lW/6wFjP4vRNrTLtUpVZ9p5bVrE2fMON/VpWA0xPJ/N5j8RVhNb8 On4jsq17n5iWcqz8uG73zvPjw4uww7QuMOP3uZTceHrOjgxDumrb8iXuvKxhtoh1qn29bqSe FFMbtWmznDd+jmpvWv9SSSZo1BFhhD5B/Rd3CCqzWgMAAA== X-Brightmail-Tracker: H4sIAAAAAAAAAzWSfUzMcRzHfX/PHcdvp/FbabjNJPPQJj6Tp/6wfvO8GYYNt/zmjrq401W2 pvSwuuqWh0TqOsWV6yJ32UJn53JHmkS5kko1okkZ3RGRK/PPe++93/u83v98GFxygwxgFMqT gkopi5ZSIkK0LTx1yYnvIfLl7WX+4BnJJKDolpmC5puVCMw1KRgMOCOhzTuI4Nez5zgU5Dcj uNrbhUONqxuBreIMBS3vpkOrZ5iChvxsClLLblHw4tMYBp0Xz2FQadkKb439BDTmlWJQMEDB lYJUzCcfMRg1mmgwJi+AvopCGsZ6Q6Gh201CfXEDCbaOxXBZ30lBna2BAFdtHwYt94oo6DaP k9DoekKAVxcIzWdzSagaKqXgk9eIg9EzTMNLuwEDl2EWVKf5qBnf/pDwONeOQca12xi0vr6P 4EFmDwYWs5uCes8gBlZLPg4/y50I+nSfaUjPGaXhSooOQXb6RQLSOsPg1w/fcvFIKKSUVBNQ 9duNNqzjzXoz4usHh3E+zRrP//S8onib10DwT0s5/m5hF82nPeigeYMljrdWhPBldQMYf/Wr h+QtpiyKt3w9R/Paz60YP9TURO8I2idac1iIVmgE1bJ1h0RybU8qdvxCZILXmksmI2e4Fvkx HLuC+/HNjE94il3ItbePTnp/dh5nze0ntUjE4Kx7KtdW/BpNFDPZjZzlbhutRQxDsAu4HlPE RCxmV3I33W/If8y5XGW1fZLj58s7susnTyVsGNdaqSfykMiAppiQv0KpiZEposOWqo/JE5WK hKVRsTEW5HsgY9LY2Vo00hLpQCyDpNPE1bZFcgkp06gTYxyIY3Cpv9hkDZZLxIdliacEVexB VVy0oHagQIaQzhZv2iMckrBHZCeFY4JwXFD9bzHGLyAZlfyeE9NlNiXMd2jWF735Mjvq+tic V0XSvfPJ3rWbL+k+DHX2z9j+4f3eoMZZ9w8UxqcEr76dszPPXvxwl/784qPx2xWu2NVRdUJm lpXT6cuDRp6XSac9YpSag/vD28dPl2RVRbSUrsFzHjubIvt3X7I7k+JWOe5s3SP+8vDMM2/W lkApoZbLQkNwlVr2F1s4Efg8AwAA X-CFilter-Loop: Reflected Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Implement CONFIG_DEPT_UNIT_TEST introducing a kernel module that runs basic unit test for dept. Signed-off-by: Byungchul Park --- include/linux/dept_unit_test.h | 67 +++++++++++ kernel/dependency/Makefile | 1 + kernel/dependency/dept.c | 12 ++ kernel/dependency/dept_unit_test.c | 173 +++++++++++++++++++++++++++++ lib/Kconfig.debug | 12 ++ 5 files changed, 265 insertions(+) create mode 100644 include/linux/dept_unit_test.h create mode 100644 kernel/dependency/dept_unit_test.c diff --git a/include/linux/dept_unit_test.h b/include/linux/dept_unit_test.h new file mode 100644 index 000000000000..7612b4e97e69 --- /dev/null +++ b/include/linux/dept_unit_test.h @@ -0,0 +1,67 @@ +// SPDX-License-Identifier: GPL-2.0+ +/* + * DEPT unit test + * + * Started by Byungchul Park : + * + * Copyright (c) 2025 SK hynix, Inc., Byungchul Park + */ + +#ifndef __LINUX_DEPT_UNIT_TEST_H +#define __LINUX_DEPT_UNIT_TEST_H + +#if defined(CONFIG_DEPT_UNIT_TEST) || defined(CONFIG_DEPT_UNIT_TEST_MODULE) +struct dept_ut { + bool circle_detected; + bool recover_circle_detected; + + int ecxt_stack_total_cnt; + int wait_stack_total_cnt; + int evnt_stack_total_cnt; + int ecxt_stack_valid_cnt; + int wait_stack_valid_cnt; + int evnt_stack_valid_cnt; +}; + +extern struct dept_ut dept_ut_results; + +static inline void dept_ut_circle_detect(void) +{ + dept_ut_results.circle_detected =3D true; +} +static inline void dept_ut_recover_circle_detect(void) +{ + dept_ut_results.recover_circle_detected =3D true; +} +static inline void dept_ut_ecxt_stack_account(bool valid) +{ + dept_ut_results.ecxt_stack_total_cnt++; + + if (valid) + dept_ut_results.ecxt_stack_valid_cnt++; +} +static inline void dept_ut_wait_stack_account(bool valid) +{ + dept_ut_results.wait_stack_total_cnt++; + + if (valid) + dept_ut_results.wait_stack_valid_cnt++; +} +static inline void dept_ut_evnt_stack_account(bool valid) +{ + dept_ut_results.evnt_stack_total_cnt++; + + if (valid) + dept_ut_results.evnt_stack_valid_cnt++; +} +#else +struct dept_ut {}; + +#define dept_ut_circle_detect() do { } while (0) +#define dept_ut_recover_circle_detect() do { } while (0) +#define dept_ut_ecxt_stack_account(v) do { } while (0) +#define dept_ut_wait_stack_account(v) do { } while (0) +#define dept_ut_evnt_stack_account(v) do { } while (0) + +#endif +#endif /* __LINUX_DEPT_UNIT_TEST_H */ diff --git a/kernel/dependency/Makefile b/kernel/dependency/Makefile index 92f165400187..fc584ca87124 100644 --- a/kernel/dependency/Makefile +++ b/kernel/dependency/Makefile @@ -2,3 +2,4 @@ =20 obj-$(CONFIG_DEPT) +=3D dept.o obj-$(CONFIG_DEPT) +=3D dept_proc.o +obj-$(CONFIG_DEPT_UNIT_TEST) +=3D dept_unit_test.o diff --git a/kernel/dependency/dept.c b/kernel/dependency/dept.c index c65bb0c6dad2..a08d0e16978b 100644 --- a/kernel/dependency/dept.c +++ b/kernel/dependency/dept.c @@ -78,8 +78,12 @@ #include #include #include +#include #include "dept_internal.h" =20 +struct dept_ut dept_ut_results; +EXPORT_SYMBOL_GPL(dept_ut_results); + static int dept_stop; static int dept_per_cpu_ready; =20 @@ -826,6 +830,10 @@ static void print_dep(struct dept_dep *d) pr_warn("(wait to wake up)\n"); print_ip_stack(0, e->ewait_stack); } + + dept_ut_ecxt_stack_account(valid_stack(e->ecxt_stack)); + dept_ut_wait_stack_account(valid_stack(w->wait_stack)); + dept_ut_evnt_stack_account(valid_stack(e->event_stack)); } } =20 @@ -920,6 +928,8 @@ static void print_circle(struct dept_class *c) dump_stack(); =20 dept_outworld_exit(); + + dept_ut_circle_detect(); } =20 /* @@ -1021,6 +1031,8 @@ static void print_recover_circle(struct dept_event_si= te *es) dump_stack(); =20 dept_outworld_exit(); + + dept_ut_recover_circle_detect(); } =20 static void bfs_init_recover(void *node, void *in, void **out) diff --git a/kernel/dependency/dept_unit_test.c b/kernel/dependency/dept_un= it_test.c new file mode 100644 index 000000000000..88e846b9f876 --- /dev/null +++ b/kernel/dependency/dept_unit_test.c @@ -0,0 +1,173 @@ +// SPDX-License-Identifier: GPL-2.0+ +/* + * DEPT unit test + * + * Started by Byungchul Park : + * + * Copyright (c) 2025 SK hynix, Inc., Byungchul Park + */ + +#include +#include +#include +#include +#include + +MODULE_DESCRIPTION("DEPT unit test"); +MODULE_LICENSE("GPL"); +MODULE_AUTHOR("Byungchul Park "); + +struct unit { + const char *name; + bool (*func)(void); + bool result; +}; + +static DEFINE_SPINLOCK(s1); +static DEFINE_SPINLOCK(s2); +static bool test_spin_lock_deadlock(void) +{ + dept_ut_results.circle_detected =3D false; + + spin_lock(&s1); + spin_lock(&s2); + spin_unlock(&s2); + spin_unlock(&s1); + + spin_lock(&s2); + spin_lock(&s1); + spin_unlock(&s1); + spin_unlock(&s2); + + return dept_ut_results.circle_detected; +} + +static DEFINE_MUTEX(m1); +static DEFINE_MUTEX(m2); +static bool test_mutex_lock_deadlock(void) +{ + dept_ut_results.circle_detected =3D false; + + mutex_lock(&m1); + mutex_lock(&m2); + mutex_unlock(&m2); + mutex_unlock(&m1); + + mutex_lock(&m2); + mutex_lock(&m1); + mutex_unlock(&m1); + mutex_unlock(&m2); + + return dept_ut_results.circle_detected; +} + +static bool test_wait_event_deadlock(void) +{ + struct dept_map dmap1; + struct dept_map dmap2; + + sdt_map_init(&dmap1); + sdt_map_init(&dmap2); + + dept_ut_results.circle_detected =3D false; + + sdt_request_event(&dmap1); /* [S] */ + sdt_wait(&dmap2); /* [W] */ + sdt_event(&dmap1); /* [E] */ + + sdt_request_event(&dmap2); /* [S] */ + sdt_wait(&dmap1); /* [W] */ + sdt_event(&dmap2); /* [E] */ + + return dept_ut_results.circle_detected; +} + +static void dummy_event(void) +{ + /* Do nothing. */ +} + +static DEFINE_DEPT_EVENT_SITE(es1); +static DEFINE_DEPT_EVENT_SITE(es2); +static bool test_recover_deadlock(void) +{ + dept_ut_results.recover_circle_detected =3D false; + + dept_recover_event(&es1, &es2); + dept_recover_event(&es2, &es1); + + event_site(&es1, dummy_event); + event_site(&es2, dummy_event); + + return dept_ut_results.recover_circle_detected; +} + +static struct unit units[] =3D { + { + .name =3D "spin lock deadlock test", + .func =3D test_spin_lock_deadlock, + }, + { + .name =3D "mutex lock deadlock test", + .func =3D test_mutex_lock_deadlock, + }, + { + .name =3D "wait event deadlock test", + .func =3D test_wait_event_deadlock, + }, + { + .name =3D "event recover deadlock test", + .func =3D test_recover_deadlock, + }, +}; + +static int __init dept_ut_init(void) +{ + int i; + + lockdep_off(); + + dept_ut_results.ecxt_stack_valid_cnt =3D 0; + dept_ut_results.ecxt_stack_total_cnt =3D 0; + dept_ut_results.wait_stack_valid_cnt =3D 0; + dept_ut_results.wait_stack_total_cnt =3D 0; + dept_ut_results.evnt_stack_valid_cnt =3D 0; + dept_ut_results.evnt_stack_total_cnt =3D 0; + + for (i =3D 0; i < ARRAY_SIZE(units); i++) + units[i].result =3D units[i].func(); + + pr_info("\n"); + pr_info("******************************************\n"); + pr_info("DEPT unit test results\n"); + pr_info("******************************************\n"); + for (i =3D 0; i < ARRAY_SIZE(units); i++) { + pr_info("(%s) %s\n", units[i].result ? "pass" : "fail", + units[i].name); + } + pr_info("ecxt stack valid count =3D %d/%d\n", + dept_ut_results.ecxt_stack_valid_cnt, + dept_ut_results.ecxt_stack_total_cnt); + pr_info("wait stack valid count =3D %d/%d\n", + dept_ut_results.wait_stack_valid_cnt, + dept_ut_results.wait_stack_total_cnt); + pr_info("event stack valid count =3D %d/%d\n", + dept_ut_results.evnt_stack_valid_cnt, + dept_ut_results.evnt_stack_total_cnt); + pr_info("******************************************\n"); + pr_info("\n"); + + lockdep_on(); + + return 0; +} + +static void dept_ut_cleanup(void) +{ + /* + * Do nothing for now. + */ +} + +module_init(dept_ut_init); +module_exit(dept_ut_cleanup); diff --git a/lib/Kconfig.debug b/lib/Kconfig.debug index ec840c672846..8db2a8136418 100644 --- a/lib/Kconfig.debug +++ b/lib/Kconfig.debug @@ -1404,6 +1404,18 @@ config DEPT_AGGRESSIVE_TIMEOUT_WAIT that timeout is used to avoid a deadlock. Say N if you'd like to avoid verbose reports. =20 +config DEPT_UNIT_TEST + tristate "unit test for DEPT" + depends on DEBUG_KERNEL && DEPT + default n + help + This option provides a kernel module that runs unit test for + DEPT. + + Say Y if you want DEPT unit test to be built into the kernel. + Say M if you want DEPT unit test to build as a module. + Say N if you are unsure. + config LOCK_DEBUGGING_SUPPORT bool depends on TRACE_IRQFLAGS_SUPPORT && STACKTRACE_SUPPORT && LOCKDEP_SUPPORT --=20 2.17.1 From nobody Fri Dec 19 15:51:27 2025 Received: from invmail4.hynix.com (exvmail4.hynix.com [166.125.252.92]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 617E1283FE2; Mon, 19 May 2025 09:19:05 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=166.125.252.92 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747646350; cv=none; b=fAdIGfATeOvniTfOlcnqd45UO25wlXivXoR++aDCdgKyqV8xNc0FnSJVBJTW/eF4nLsqhsxE4jYbnGMCUQtIuodatYZZHg2CqmCqCAYFy1O2f6HP47VhSv02/SpZ8RFGpYqCO7zG2RLxjgPqKCxdQYeuEvgBzsP3fnGDY7NuUoQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747646350; c=relaxed/simple; bh=Kn0pCpXycQGzli+Ttb42Y8sONcxizenmqlKSwqT8CyA=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References; b=YQVBEYzKp8LqHn0sKp7LWY7gYIFqutW+dsSXhBi3fEjas/6P+rS1C9RAA/t52pfK3d+KeqP7X3NM3aXWTRcAbR/fgVOy0OLs58rVl8rmtGllUo8KBA4YOOiDEyPo7Kp1hS/3097N12wYcEK/NI6LZbEOvO+P1iZhQ4dl95QPJfw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com; spf=pass smtp.mailfrom=sk.com; arc=none smtp.client-ip=166.125.252.92 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=sk.com X-AuditID: a67dfc5b-669ff7000002311f-63-682af77127a4 From: Byungchul Park To: linux-kernel@vger.kernel.org Cc: kernel_team@skhynix.com, torvalds@linux-foundation.org, damien.lemoal@opensource.wdc.com, linux-ide@vger.kernel.org, adilger.kernel@dilger.ca, linux-ext4@vger.kernel.org, mingo@redhat.com, peterz@infradead.org, will@kernel.org, tglx@linutronix.de, rostedt@goodmis.org, joel@joelfernandes.org, sashal@kernel.org, daniel.vetter@ffwll.ch, duyuyang@gmail.com, johannes.berg@intel.com, tj@kernel.org, tytso@mit.edu, willy@infradead.org, david@fromorbit.com, amir73il@gmail.com, gregkh@linuxfoundation.org, kernel-team@lge.com, linux-mm@kvack.org, akpm@linux-foundation.org, mhocko@kernel.org, minchan@kernel.org, hannes@cmpxchg.org, vdavydov.dev@gmail.com, sj@kernel.org, jglisse@redhat.com, dennis@kernel.org, cl@linux.com, penberg@kernel.org, rientjes@google.com, vbabka@suse.cz, ngupta@vflare.org, linux-block@vger.kernel.org, josef@toxicpanda.com, linux-fsdevel@vger.kernel.org, jack@suse.cz, jlayton@kernel.org, dan.j.williams@intel.com, hch@infradead.org, djwong@kernel.org, dri-devel@lists.freedesktop.org, rodrigosiqueiramelo@gmail.com, melissa.srw@gmail.com, hamohammed.sa@gmail.com, harry.yoo@oracle.com, chris.p.wilson@intel.com, gwan-gyeong.mun@intel.com, max.byungchul.park@gmail.com, boqun.feng@gmail.com, longman@redhat.com, yskelg@gmail.com, yunseong.kim@ericsson.com, yeoreum.yun@arm.com, netdev@vger.kernel.org, matthew.brost@intel.com, her0gyugyu@gmail.com Subject: [PATCH v16 42/42] dept: call dept_hardirqs_off() in local_irq_*() regardless of irq state Date: Mon, 19 May 2025 18:18:26 +0900 Message-Id: <20250519091826.19752-43-byungchul@sk.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20250519091826.19752-1-byungchul@sk.com> References: <20250519091826.19752-1-byungchul@sk.com> X-Brightmail-Tracker: H4sIAAAAAAAAAzWSW0wTaRTH/b6Zzkwr1bG67iyQaEqIiiuK0fVEXWNi1HnxkgCJSoLWZbJt BNRWUVZNQJBFsIgYqIBgKVorFMWWB29VFgTFC1ZlEQhWwA3YcGmstNqCu9t6eTn55X/O+Z2X wxCyB6JQRpV6QFCnKpLllISUjIZULdr/MUq5xOdfAZ7xXBLOXzNTYL9ai8DckInB2bIRXnlH EEw8fUaArtiOoKr/NQENrQ4ENtNxCl7+Mw06PC4K2orzKciqvkbB8+FJDL0lRRhqLZvgjXGQ hMeFBgw6JwXluiwcKO8w+Iw1NBgzImHAVEbDZH8MtDk6RWDrWQillb0U3LG1kdB6YwDDy1vn KXCY/xPB49aHJHgLwsB+RiuCujEDBcNeIwFGj4uGF416DK362VCfHRDmfPhXBA+0jRhyLl7H 0NF9G8Hd3D4MFnMnBc2eEQxWSzEB/sstCAYKRmk4ccpHQ3lmAYL8EyUkZPcuh4lPgcsV4zGQ eaGehLrPnWjtr7y50oz45hEXwWdbD/F+z98Ub/PqSf6RgeNvlr2m+ey7PTSvtxzkraYovvqO E/NVbo+It9ScpHiLu4jm80Y7MD/W3k5vDd8hWZ0kJKvSBPXiNbskyhKTidyXOf3wk3N+IgON Ts1DYoZjl3E2w1vyO/eNnRYFmWLncV1dPiLIs9i5nFU7GMglDMF2TuVeVXSjPMQwM9nd3MjH kOAMyUZyxWeHcZCl7C9cnc6JvzrncLX1jV884kDek9+Mgixjl3MdtZVk0Mmx1WLu+FD7t4Wf uL9MXWQhkurRlBokU6WmpShUycuilempqsPRv+1NsaDAexmPTSbcQG57bBNiGSQPkdbbFihl IkWaJj2lCXEMIZ8lrbHOV8qkSYr0PwT13p3qg8mCpgmFMaT8R+lS76EkGfu74oCwRxD2Cerv XcyIQzMQCk0wZ5g+RazbEL8ozB1R5vJuW53f122vII64Lzkd85aEDzY44iY8fzpaok9eiU6c ErdfdyVn08Ltimloj2HleP9Qaaxjy9HNjUU+bXjDinjXQ5P+3tz1mzXhC3Z5435IvL9FW5gr fxt7Nt5/b+YbSdbQ+xmrLtE/f4bCCHF7ORMpJzVKRUwUodYo/geZXTjvWgMAAA== X-Brightmail-Tracker: H4sIAAAAAAAAAzWSe0yTZxSH977flUr1W0f0i8zgmhARohOyuuNlTJMlviPRuEgwMfFS9Iv9 YkFplcniBUZ1CMLQpSAorKKplRZlrZtaqSEQwKoTFKyKgEIWlbSURCkO7C6ty/45efL7nTzn n8NTqovMXF7O3SsZcrV6NaugFetWFC/Ke5usW/K07WMITZTQcOayg4WeS3YEjitFGEY71sCj yQCCd793U1Bt7kFwdniQgiudQwg8tu9Z6P1jJvSFxlnwmstYKD53mYX7/jCGgaqTGOzOtfDM +oKGO5UNGKpHWThdXYwj4xWGKWsjB9bCRBix1XIQHk4F75CPgfY6LwOe/hSoqR9gocXjpaHz 2giGXvcZFoYc/zBwp/MWDZMV8dBzopyBpmADC/5JKwXW0DgHD1otGDots6HZFLEeffM3A13l rRiOnv8FQ9+TGwhuljzH4HT4WGgPBTC4nGYKpi90IBipGOPgyPEpDk4XVSAoO1JFg2lAA+/+ jFyum0iFop+baWj6y4dWpRNHvQOR9sA4RUyub8l06CFLPJMWmtxuEMn12kGOmG72c8Ti3Edc tmRyrmUUk7OvQwxxNh5jifP1SY6UjvVhErx3j1s/b5Ni5Q5JL+dLhk/Ttyl0VTYbvado1v67 p6apQjQ2oxTF8KLwmfg8+CMTZVZYID5+PEVFOU6YL7rKX0RyBU8Jvhnio7onqBTx/EdCthh4 GxvdoYVE0fyTH0dZKSwVm6pH8X/OBNHe3PreExPJ+8vaUZRVgkbss9fTlUhhQR80ojg5Nz9H K+s1i427dAW58v7F23fnOFHkgawHwyeuoYneNW1I4JE6VtnsWahTMdp8Y0FOGxJ5Sh2nbHQl 6VTKHdqC7yTD7q2GfXrJ2IbieVo9R5mxUdqmEnZq90q7JGmPZPi/xXzM3ELkMGd0pSVmvjke JgFfxup4r7nQLidkqot/SNfZ85b7tZVfsocr3N1Z5o5u/EnivJSFL7eslE3fpCjkQ5qu0tjP Sw6Xx141rqvxa1Zl/5ow/Nug3j2z9sHBvGx3zX2rvCAtb8j2dVowM3wg6YvNWV+1XOpyBq3T hxwvD3y4gVnGuNW0UadNTaYMRu2/Yfj0JDwDAAA= X-CFilter-Loop: Reflected Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" For dept to function properly, dept_task()->hardirqs_enabled must be set correctly. If it fails to set this value to false, for example, dept may mistakenly think irq is still enabled even when it's not. Do dept_hardirqs_off() regardless of irq state not to miss any unexpected cases by any chance e.g. changes of the state by asm code. Signed-off-by: Byungchul Park --- include/linux/irqflags.h | 14 ++++++++++++++ kernel/dependency/dept.c | 1 + 2 files changed, 15 insertions(+) diff --git a/include/linux/irqflags.h b/include/linux/irqflags.h index d8b9cf093f83..586f5bad4da7 100644 --- a/include/linux/irqflags.h +++ b/include/linux/irqflags.h @@ -214,6 +214,13 @@ extern void warn_bogus_irq_restore(void); raw_local_irq_disable(); \ if (!was_disabled) \ trace_hardirqs_off(); \ + /* \ + * Just in case that C code has missed \ + * trace_hardirqs_off() at the first \ + * place e.g. disabling irq at asm code.\ + */ \ + else \ + dept_hardirqs_off(); \ } while (0) =20 #define local_irq_save(flags) \ @@ -221,6 +228,13 @@ extern void warn_bogus_irq_restore(void); raw_local_irq_save(flags); \ if (!raw_irqs_disabled_flags(flags)) \ trace_hardirqs_off(); \ + /* \ + * Just in case that C code has missed \ + * trace_hardirqs_off() at the first \ + * place e.g. disabling irq at asm code.\ + */ \ + else \ + dept_hardirqs_off(); \ } while (0) =20 #define local_irq_restore(flags) \ diff --git a/kernel/dependency/dept.c b/kernel/dependency/dept.c index a08d0e16978b..4ca1cc04293c 100644 --- a/kernel/dependency/dept.c +++ b/kernel/dependency/dept.c @@ -2248,6 +2248,7 @@ void noinstr dept_hardirqs_off(void) */ dept_task()->hardirqs_enabled =3D false; } +EXPORT_SYMBOL_GPL(dept_hardirqs_off); =20 void noinstr dept_update_cxt(void) { --=20 2.17.1