From nobody Sun Sep 14 18:26:45 2025 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4C550C38142 for ; Thu, 19 Jan 2023 04:41:55 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230357AbjASEly (ORCPT ); Wed, 18 Jan 2023 23:41:54 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:56826 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229714AbjASElD (ORCPT ); Wed, 18 Jan 2023 23:41:03 -0500 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 40B214955C for ; Wed, 18 Jan 2023 20:38:38 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1674103023; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=YFRn2G3kICVUTv0CVX/GON7X4LfKNre2jK8Ma2QSF+o=; b=e1ae3WS1lV4OMkKeGOBbwcORYIT2vCgoD3h3XApGYO2Y84mcJa0kozIOLX1AWciHhKFQ4M J8cIEa6kBc0Y2Vv49nRF+4tpAfESfAu/xP7D105DbEdLJFh4yCzIc6tZdnEvIRzC9DUlQa TfD5qNfmL8y0B+Uob25BasUbvwhEOYc= Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-584-1DQLsC-QPwm5vFLMtYRjPg-1; Wed, 18 Jan 2023 23:01:21 -0500 X-MC-Unique: 1DQLsC-QPwm5vFLMtYRjPg-1 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.rdu2.redhat.com [10.11.54.5]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 0E2E53814950; Thu, 19 Jan 2023 04:01:21 +0000 (UTC) Received: from llong.com (unknown [10.22.8.15]) by smtp.corp.redhat.com (Postfix) with ESMTP id B6CE551E5; Thu, 19 Jan 2023 04:01:20 +0000 (UTC) From: Waiman Long To: Catalin Marinas , Andrew Morton Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org, Muchun Song , Waiman Long Subject: [RESEND PATCH v2 1/2] mm/kmemleak: Simplify kmemleak_cond_resched() usage Date: Wed, 18 Jan 2023 23:01:10 -0500 Message-Id: <20230119040111.350923-2-longman@redhat.com> In-Reply-To: <20230119040111.350923-1-longman@redhat.com> References: <20230119040111.350923-1-longman@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.1 on 10.11.54.5 Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" The presence of a pinned argument and the 64k loop count make kmemleak_cond_resched() a bit more complex to read. The pinned argument is used only by first kmemleak_scan() loop. Simplify the usage of kmemleak_cond_resched() by removing the pinned argument and always do a get_object()/put_object() sequence. In addition, the 64k loop is removed by using need_resched() to decide if kmemleak_cond_resched() should be called. Signed-off-by: Waiman Long Reviewed-by: Catalin Marinas --- mm/kmemleak.c | 48 ++++++++++++------------------------------------ 1 file changed, 12 insertions(+), 36 deletions(-) diff --git a/mm/kmemleak.c b/mm/kmemleak.c index 267332904354..e7cb521236bf 100644 --- a/mm/kmemleak.c +++ b/mm/kmemleak.c @@ -1463,22 +1463,17 @@ static void scan_gray_list(void) /* * Conditionally call resched() in an object iteration loop while making s= ure * that the given object won't go away without RCU read lock by performing= a - * get_object() if !pinned. - * - * Return: false if can't do a cond_resched() due to get_object() failure - * true otherwise + * get_object() if necessaary. */ -static bool kmemleak_cond_resched(struct kmemleak_object *object, bool pin= ned) +static void kmemleak_cond_resched(struct kmemleak_object *object) { - if (!pinned && !get_object(object)) - return false; + if (!get_object(object)) + return; /* Try next object */ =20 rcu_read_unlock(); cond_resched(); rcu_read_lock(); - if (!pinned) - put_object(object); - return true; + put_object(object); } =20 /* @@ -1492,15 +1487,12 @@ static void kmemleak_scan(void) struct zone *zone; int __maybe_unused i; int new_leaks =3D 0; - int loop_cnt =3D 0; =20 jiffies_last_scan =3D jiffies; =20 /* prepare the kmemleak_object's */ rcu_read_lock(); list_for_each_entry_rcu(object, &object_list, object_list) { - bool obj_pinned =3D false; - raw_spin_lock_irq(&object->lock); #ifdef DEBUG /* @@ -1526,19 +1518,13 @@ static void kmemleak_scan(void) =20 /* reset the reference count (whiten the object) */ object->count =3D 0; - if (color_gray(object) && get_object(object)) { + if (color_gray(object) && get_object(object)) list_add_tail(&object->gray_list, &gray_list); - obj_pinned =3D true; - } =20 raw_spin_unlock_irq(&object->lock); =20 - /* - * Do a cond_resched() every 64k objects to avoid soft lockup. - */ - if (!(++loop_cnt & 0xffff) && - !kmemleak_cond_resched(object, obj_pinned)) - loop_cnt--; /* Try again on next object */ + if (need_resched()) + kmemleak_cond_resched(object); } rcu_read_unlock(); =20 @@ -1605,14 +1591,9 @@ static void kmemleak_scan(void) * scan and color them gray until the next scan. */ rcu_read_lock(); - loop_cnt =3D 0; list_for_each_entry_rcu(object, &object_list, object_list) { - /* - * Do a cond_resched() every 64k objects to avoid soft lockup. - */ - if (!(++loop_cnt & 0xffff) && - !kmemleak_cond_resched(object, false)) - loop_cnt--; /* Try again on next object */ + if (need_resched()) + kmemleak_cond_resched(object); =20 /* * This is racy but we can save the overhead of lock/unlock @@ -1647,14 +1628,9 @@ static void kmemleak_scan(void) * Scanning result reporting. */ rcu_read_lock(); - loop_cnt =3D 0; list_for_each_entry_rcu(object, &object_list, object_list) { - /* - * Do a cond_resched() every 64k objects to avoid soft lockup. - */ - if (!(++loop_cnt & 0xffff) && - !kmemleak_cond_resched(object, false)) - loop_cnt--; /* Try again on next object */ + if (need_resched()) + kmemleak_cond_resched(object); =20 /* * This is racy but we can save the overhead of lock/unlock --=20 2.31.1 From nobody Sun Sep 14 18:26:45 2025 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7F7C4C38142 for ; Thu, 19 Jan 2023 04:41:19 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229656AbjASElR (ORCPT ); Wed, 18 Jan 2023 23:41:17 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:54776 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230161AbjASEkb (ORCPT ); Wed, 18 Jan 2023 23:40:31 -0500 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 20248457E2 for ; Wed, 18 Jan 2023 20:37:29 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1674102975; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Eek5YPddR+rdvDYmWL3WCwvTt/kh6fJNelSHfFXusYg=; b=d89wGSWqvQEBrDDRsk0a5ELPYCM/db16cQznQOCU00ym9zDzPunXE9ToV9nPrT/CYsS+5W LiVhrN1UhgF5YR7Xxrbb3cvks97GxQshleEpLLfrKaDh1fahVCIBauCp3MOD43Q3IK0QfC UNK9uY9qDvGPPh0WC6Op5VSY6s/a1VU= Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-550-JseQALo8MkyN83fBd0phlQ-1; Wed, 18 Jan 2023 23:01:21 -0500 X-MC-Unique: JseQALo8MkyN83fBd0phlQ-1 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.rdu2.redhat.com [10.11.54.5]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 657E52A5956F; Thu, 19 Jan 2023 04:01:21 +0000 (UTC) Received: from llong.com (unknown [10.22.8.15]) by smtp.corp.redhat.com (Postfix) with ESMTP id 1C65751E5; Thu, 19 Jan 2023 04:01:21 +0000 (UTC) From: Waiman Long To: Catalin Marinas , Andrew Morton Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org, Muchun Song , Waiman Long Subject: [RESEND PATCH v2 2/2] mm/kmemleak: Fix UAF bug in kmemleak_scan() Date: Wed, 18 Jan 2023 23:01:11 -0500 Message-Id: <20230119040111.350923-3-longman@redhat.com> In-Reply-To: <20230119040111.350923-1-longman@redhat.com> References: <20230119040111.350923-1-longman@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.1 on 10.11.54.5 Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Commit 6edda04ccc7c ("mm/kmemleak: prevent soft lockup in first object iteration loop of kmemleak_scan()") fixes soft lockup problem in kmemleak_scan() by periodically doing a cond_resched(). It does take a reference of the current object before doing it. Unfortunately, if the object has been deleted from the object_list, the next object pointed to by its next pointer may no longer be valid after coming back from cond_resched(). This can result in use-after-free and other nasty problem. Fix this problem by adding a del_state flag into kmemleak_object structure to synchronize the object deletion process between kmemleak_cond_resched() and __remove_object() to make sure that the object remained in the object_list in the duration of the cond_resched() call. Fixes: 6edda04ccc7c ("mm/kmemleak: prevent soft lockup in first object iter= ation loop of kmemleak_scan()") Signed-off-by: Waiman Long Reviewed-by: Catalin Marinas --- mm/kmemleak.c | 35 +++++++++++++++++++++++++++++------ 1 file changed, 29 insertions(+), 6 deletions(-) diff --git a/mm/kmemleak.c b/mm/kmemleak.c index e7cb521236bf..0ece170fc9ef 100644 --- a/mm/kmemleak.c +++ b/mm/kmemleak.c @@ -13,11 +13,12 @@ * * The following locks and mutexes are used by kmemleak: * - * - kmemleak_lock (raw_spinlock_t): protects the object_list modification= s and - * accesses to the object_tree_root (or object_phys_tree_root). The - * object_list is the main list holding the metadata (struct kmemleak_ob= ject) - * for the allocated memory blocks. The object_tree_root and object_phys= _tree_root - * are red black trees used to look-up metadata based on a pointer to the + * - kmemleak_lock (raw_spinlock_t): protects the object_list as well as + * del_state modifications and accesses to the object_tree_root (or + * object_phys_tree_root). The object_list is the main list holding the + * metadata (struct kmemleak_object) for the allocated memory blocks. + * The object_tree_root and object_phys_tree_root are red + * black trees used to look-up metadata based on a pointer to the * corresponding memory block. The object_phys_tree_root is for objects * allocated with physical address. The kmemleak_object structures are * added to the object_list and object_tree_root (or object_phys_tree_ro= ot) @@ -147,6 +148,7 @@ struct kmemleak_object { struct rcu_head rcu; /* object_list lockless traversal */ /* object usage count; object freed when use_count =3D=3D 0 */ atomic_t use_count; + unsigned int del_state; /* deletion state */ unsigned long pointer; size_t size; /* pass surplus references to this pointer */ @@ -177,6 +179,11 @@ struct kmemleak_object { /* flag set for object allocated with physical address */ #define OBJECT_PHYS (1 << 4) =20 +/* set when __remove_object() called */ +#define DELSTATE_REMOVED (1 << 0) +/* set to temporarily prevent deletion from object_list */ +#define DELSTATE_NO_DELETE (1 << 1) + #define HEX_PREFIX " " /* number of bytes to print per line; must be 16 or 32 */ #define HEX_ROW_SIZE 16 @@ -567,7 +574,9 @@ static void __remove_object(struct kmemleak_object *obj= ect) rb_erase(&object->rb_node, object->flags & OBJECT_PHYS ? &object_phys_tree_root : &object_tree_root); - list_del_rcu(&object->object_list); + if (!(object->del_state & DELSTATE_NO_DELETE)) + list_del_rcu(&object->object_list); + object->del_state |=3D DELSTATE_REMOVED; } =20 /* @@ -633,6 +642,7 @@ static void __create_object(unsigned long ptr, size_t s= ize, object->count =3D 0; /* white color initially */ object->jiffies =3D jiffies; object->checksum =3D 0; + object->del_state =3D 0; =20 /* task information */ if (in_hardirq()) { @@ -1470,9 +1480,22 @@ static void kmemleak_cond_resched(struct kmemleak_ob= ject *object) if (!get_object(object)) return; /* Try next object */ =20 + raw_spin_lock_irq(&kmemleak_lock); + if (object->del_state & DELSTATE_REMOVED) + goto unlock_put; /* Object removed */ + object->del_state |=3D DELSTATE_NO_DELETE; + raw_spin_unlock_irq(&kmemleak_lock); + rcu_read_unlock(); cond_resched(); rcu_read_lock(); + + raw_spin_lock_irq(&kmemleak_lock); + if (object->del_state & DELSTATE_REMOVED) + list_del_rcu(&object->object_list); + object->del_state &=3D ~DELSTATE_NO_DELETE; +unlock_put: + raw_spin_unlock_irq(&kmemleak_lock); put_object(object); } =20 --=20 2.31.1