From nobody Sat Nov 30 16:42:11 2024 Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.15]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A47F312B64 for ; Mon, 9 Sep 2024 01:30:07 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=198.175.65.15 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1725845409; cv=none; b=gVDNz3ySdyZ4XeonpH90DEnrnHCLjaNemS/vh04d0mAiGmXVy8QS29Qw2RrF1PyCgIFZLQ6jYAeuLznzlfMIVqA1RtADJgV+/33wk0CzijZI7EBivGfLkTtsuJt79H54ahrvxKCyoGa3o1Jj5F6t2dfsKLAzWBgKIySXTC+B7kQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1725845409; c=relaxed/simple; bh=pVsgqwZadHhdmsZUe9yKROnQBO6McnXZ8J3vApLU53I=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=uy1Eaw6eJVJBLkEcHVJIi+IDABKn9z6zI4Cl6DDec3zmL1pqcdjIgxlOfCkViTCZp0Wvi2vbLlvLYIrO1T4SQDa2ODARHDMyGoMdDhLd4+VhOYNMK2KCl6hgtXeH+qdgs+ENKCWvfLOlwwk0cLV0rvMGKJRPcCNba9NUcOxtJ50= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=T2sx01ob; arc=none smtp.client-ip=198.175.65.15 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="T2sx01ob" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1725845408; x=1757381408; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=pVsgqwZadHhdmsZUe9yKROnQBO6McnXZ8J3vApLU53I=; b=T2sx01obrcg6hn4YcJPB0+HLOSHNPzDxDRbu8GUa5zptODvli9Tp8q5o VXyTupaKHHxrFOBU9zJxE9Jfdcvs2CH76SsiUF/uSvRjgpyAQ5Tr/zE6S pXR6qkmjl2xDe7i0YT7CfJieK3G5Vtbg4SU1lnh9TcLamZREHiasEYFf3 1qmRcaR88nsPAy6rYpgdpH9XbVqbV0plNBwzUJG4xqYUbosKzntNkHj8X pW/Md9qD0fu/jgm0Gtn/Izaa3wRtEG0FBp+UVUq715ml0GMpTGLR5AGjp wm7Oedb1+vN9QOPu/I2V1RpxgXZdhrtKUna0XpNqklcobfPZPUd8kdNzX g==; X-CSE-ConnectionGUID: n82M5v+1QYuF/TAs1FYRnQ== X-CSE-MsgGUID: 5BGZMdDjSl674Fpwe3V8sA== X-IronPort-AV: E=McAfee;i="6700,10204,11189"; a="28258099" X-IronPort-AV: E=Sophos;i="6.10,213,1719903600"; d="scan'208";a="28258099" Received: from orviesa009.jf.intel.com ([10.64.159.149]) by orvoesa107.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 08 Sep 2024 18:30:07 -0700 X-CSE-ConnectionGUID: UowcQFlWTN6ZxGnlayNrVA== X-CSE-MsgGUID: NSxbVtALS32RTELxfc1T0A== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.10,213,1719903600"; d="scan'208";a="66486438" Received: from feng-clx.sh.intel.com ([10.239.159.50]) by orviesa009.jf.intel.com with ESMTP; 08 Sep 2024 18:30:03 -0700 From: Feng Tang To: Vlastimil Babka , Andrew Morton , Christoph Lameter , Pekka Enberg , David Rientjes , Joonsoo Kim , Roman Gushchin , Hyeonggon Yoo <42.hyeyoo@gmail.com>, Andrey Konovalov , Marco Elver , Shuah Khan , David Gow , Danilo Krummrich Cc: linux-mm@kvack.org, kasan-dev@googlegroups.com, linux-kernel@vger.kernel.org, Feng Tang Subject: [PATCH 1/5] mm/kasan: Don't store metadata inside kmalloc object when slub_debug_orig_size is on Date: Mon, 9 Sep 2024 09:29:54 +0800 Message-Id: <20240909012958.913438-2-feng.tang@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240909012958.913438-1-feng.tang@intel.com> References: <20240909012958.913438-1-feng.tang@intel.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" For a kmalloc object, when both kasan and slub redzone sanity check are enabled, they could both manipulate its data space like storing kasan free meta data and setting up kmalloc redzone, and may affect accuracy of that object's 'orig_size'. As an accurate 'orig_size' will be needed by some function like krealloc() soon, save kasan's free meta data in slub's metadata area instead of inside object when 'orig_size' is enabled.=20 This will make it easier to maintain/understand the code. Size wise, when these two options are both enabled, the slub meta data space is already huge, and this just slightly increase the overall size. Signed-off-by: Feng Tang Acked-by: Andrey Konovalov --- mm/kasan/generic.c | 5 ++++- mm/slab.h | 6 ++++++ mm/slub.c | 17 ----------------- 3 files changed, 10 insertions(+), 18 deletions(-) diff --git a/mm/kasan/generic.c b/mm/kasan/generic.c index 6310a180278b..cad376199d47 100644 --- a/mm/kasan/generic.c +++ b/mm/kasan/generic.c @@ -393,8 +393,11 @@ void kasan_cache_create(struct kmem_cache *cache, unsi= gned int *size, * be touched after it was freed, or * 2. Object has a constructor, which means it's expected to * retain its content until the next allocation. + * 3. It is from a kmalloc cache which enables the debug option + * to store original size. */ - if ((cache->flags & SLAB_TYPESAFE_BY_RCU) || cache->ctor) { + if ((cache->flags & SLAB_TYPESAFE_BY_RCU) || cache->ctor || + slub_debug_orig_size(cache)) { cache->kasan_info.free_meta_offset =3D *size; *size +=3D sizeof(struct kasan_free_meta); goto free_meta_added; diff --git a/mm/slab.h b/mm/slab.h index 90f95bda4571..7a0e9b34ba2a 100644 --- a/mm/slab.h +++ b/mm/slab.h @@ -689,6 +689,12 @@ void __kmem_obj_info(struct kmem_obj_info *kpp, void *= object, struct slab *slab) void __check_heap_object(const void *ptr, unsigned long n, const struct slab *slab, bool to_user); =20 +static inline bool slub_debug_orig_size(struct kmem_cache *s) +{ + return (kmem_cache_debug_flags(s, SLAB_STORE_USER) && + (s->flags & SLAB_KMALLOC)); +} + #ifdef CONFIG_SLUB_DEBUG void skip_orig_size_check(struct kmem_cache *s, const void *object); #endif diff --git a/mm/slub.c b/mm/slub.c index 23761533329d..996a72fa6f62 100644 --- a/mm/slub.c +++ b/mm/slub.c @@ -230,12 +230,6 @@ static inline bool kmem_cache_debug(struct kmem_cache = *s) return kmem_cache_debug_flags(s, SLAB_DEBUG_FLAGS); } =20 -static inline bool slub_debug_orig_size(struct kmem_cache *s) -{ - return (kmem_cache_debug_flags(s, SLAB_STORE_USER) && - (s->flags & SLAB_KMALLOC)); -} - void *fixup_red_left(struct kmem_cache *s, void *p) { if (kmem_cache_debug_flags(s, SLAB_RED_ZONE)) @@ -760,21 +754,10 @@ static inline void set_orig_size(struct kmem_cache *s, void *object, unsigned int orig_size) { void *p =3D kasan_reset_tag(object); - unsigned int kasan_meta_size; =20 if (!slub_debug_orig_size(s)) return; =20 - /* - * KASAN can save its free meta data inside of the object at offset 0. - * If this meta data size is larger than 'orig_size', it will overlap - * the data redzone in [orig_size+1, object_size]. Thus, we adjust - * 'orig_size' to be as at least as big as KASAN's meta data. - */ - kasan_meta_size =3D kasan_metadata_size(s, true); - if (kasan_meta_size > orig_size) - orig_size =3D kasan_meta_size; - p +=3D get_info_end(s); p +=3D sizeof(struct track) * 2; =20 --=20 2.34.1 From nobody Sat Nov 30 16:42:11 2024 Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.15]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9FEE718AEA for ; Mon, 9 Sep 2024 01:30:11 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=198.175.65.15 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1725845413; cv=none; b=XMty7dsgFKSzMJgWQETLJIDOJNuPgI5pKIHBc4/6Hjxbe0Xu6wIFUMEFew0mxDzqn4ZH/5H5700vVdcmikP0sD3b2punQKvoHZLjyRGXIf/EK8pwMfCWmaXKNj2rA5Go6uAhxDTgTCJnWEyY2w4gr79/8ndRn36GKg5yeqiMk4Q= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1725845413; c=relaxed/simple; bh=XQhb14NwobzgpHBJToS+EBksbjOmwkl1vy/zAbUzlvA=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=pkp/Bn7cAiuUmPpPEzfpgiauehAtCmtH6V62S9uIdOhhyJAWR5n5f4ey/R0/79a0AnpUOWAD/6dbLU4zZ97uZTKIoHEXc3FyB/8cZzaQebFntV+hXBoH/xH0A5x8CCxAlq3VmvrdrpMII2hXhsoIgHWeyRCcgqQDsWaMS3MqX/8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=jDz1jhGK; arc=none smtp.client-ip=198.175.65.15 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="jDz1jhGK" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1725845412; x=1757381412; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=XQhb14NwobzgpHBJToS+EBksbjOmwkl1vy/zAbUzlvA=; b=jDz1jhGKJRnFkH03NhfTiJFMUEeUKuQN+DywzQYCi97G1/3rDoI9F0OC fchBMfq/m+xf+Fq10auRWA0nI6TYwrfQupIkZh7NntObMpNWtadoNobBP PMLkTXus+s2cwDuSW6xicI+FqG+1/IUfcb8oGveUV33vSKDiZisVTMhhD tdIcwFVTgdnZUYVlKW/I+2P1dPX9ZVVkDpVnMQWClQLtYtm88xnZpwvCo LeBH3YRR1WUgt2lKnYx2fGNmjp7D1af8TfhoMhDCjba2/WsNdOXqg/OdN yyO8qx7Iv0MMXf/6RDkX8xKcTY8jiL7fZGXW3yT8ToOmYQBzo9e+Vj4jE A==; X-CSE-ConnectionGUID: QPrWmUqvRcyovYxwCx7YiQ== X-CSE-MsgGUID: 74ZVgdGhTaqa4g5jb70Fcw== X-IronPort-AV: E=McAfee;i="6700,10204,11189"; a="28258114" X-IronPort-AV: E=Sophos;i="6.10,213,1719903600"; d="scan'208";a="28258114" Received: from orviesa009.jf.intel.com ([10.64.159.149]) by orvoesa107.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 08 Sep 2024 18:30:11 -0700 X-CSE-ConnectionGUID: DjU4Ii44RO+mC+79SYUM9Q== X-CSE-MsgGUID: wZvquO98SZKlH1AaBHu3rw== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.10,213,1719903600"; d="scan'208";a="66486450" Received: from feng-clx.sh.intel.com ([10.239.159.50]) by orviesa009.jf.intel.com with ESMTP; 08 Sep 2024 18:30:07 -0700 From: Feng Tang To: Vlastimil Babka , Andrew Morton , Christoph Lameter , Pekka Enberg , David Rientjes , Joonsoo Kim , Roman Gushchin , Hyeonggon Yoo <42.hyeyoo@gmail.com>, Andrey Konovalov , Marco Elver , Shuah Khan , David Gow , Danilo Krummrich Cc: linux-mm@kvack.org, kasan-dev@googlegroups.com, linux-kernel@vger.kernel.org, Feng Tang Subject: [PATCH 2/5] mm/slub: Consider kfence case for get_orig_size() Date: Mon, 9 Sep 2024 09:29:55 +0800 Message-Id: <20240909012958.913438-3-feng.tang@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240909012958.913438-1-feng.tang@intel.com> References: <20240909012958.913438-1-feng.tang@intel.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" When 'orig_size' of kmalloc object is enabled by debug option, it should either contains the actual requested size or the cache's 'object_size'. But it's not true if that object is a kfence-allocated one, and its 'orig_size' in metadata could be zero or other values. This is not a big issue for current 'orig_size' usage, as init_object() and check_object() during alloc/free process will be skipped for kfence addresses. As 'orig_size' will be used by some function block like krealloc(), handle it by returning the 'object_size' in get_orig_size() for kfence addresses. Signed-off-by: Feng Tang --- mm/slub.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/mm/slub.c b/mm/slub.c index 996a72fa6f62..4cb3822dba08 100644 --- a/mm/slub.c +++ b/mm/slub.c @@ -768,7 +768,7 @@ static inline unsigned int get_orig_size(struct kmem_ca= che *s, void *object) { void *p =3D kasan_reset_tag(object); =20 - if (!slub_debug_orig_size(s)) + if (!slub_debug_orig_size(s) || is_kfence_address(object)) return s->object_size; =20 p +=3D get_info_end(s); --=20 2.34.1 From nobody Sat Nov 30 16:42:11 2024 Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.15]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id EE6401BC46 for ; Mon, 9 Sep 2024 01:30:15 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=198.175.65.15 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1725845417; cv=none; b=LGSTvyCC5uqsU9NcTHpD3TKXwjmO2VzqN9gdOMTdKfg4+tDDV5rW+rM6UOvPiHT2u74dTfwmHQqlBa9JiMPlnqzPtJqpJ7A7TcgQRFBiqewGBrCrJZCCC2kBo5uGLKOnvMpPPKtMy74vGieiJjY4NeYQ9Gqcg8FLoCXiby3nEx8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1725845417; c=relaxed/simple; bh=B8mgHFZzGD1KfDB0WGMvxSgxbcgttavsuRFve7SsS4M=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=V0/PdccIaiUjFZABfBRQ8aaFeSDtynuCIn1ngJfZ0YoSUIYmkH+i5AuxbfRiv27hghCAKcCJvPsFBx4NNqCRUVflv0xQZKhRqyHkZBlmqkMlxX7gCsVvHz7Qyqqo196P92+/AdZ2vMFks3CpGw/nnnz6yw4rr19R9LCw55KOfC0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=UO0zbAn1; arc=none smtp.client-ip=198.175.65.15 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="UO0zbAn1" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1725845416; x=1757381416; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=B8mgHFZzGD1KfDB0WGMvxSgxbcgttavsuRFve7SsS4M=; b=UO0zbAn1j7vuzTnpmStJmkvfnOxNJ9Op+nqvh7P0xugVc96RZGWWASwb z/gE5KNpIX17z6sneuDNeuA3JaBXKEQIgzo7l8xqkQUFH4b/3mVC3dNkV OdJ5tM0lu7R912jZUzxPKYiiCB+6PQjPBSpJIsFeTDt5jzctY2JuSj9x5 Sda12nZMHLEHb0srYy2neudfWm2JQgmCOLuG/XRugpIIUJ7ZvdV0Oiaqp S+wza6LXdpf+iZ/2XOKkTzcetf1LbtoowfwjCIjCkA2mYR++ZOy0RfM2k hGc5mnrWQm2iuJqGNyLznY592COpxz8ONuNCH80Wmm8bIUKD7SPp6X9EA g==; X-CSE-ConnectionGUID: RTSnSDEjRSuRD8YvWcjSEA== X-CSE-MsgGUID: 18M8fqxXSZOpgnsjXniTzQ== X-IronPort-AV: E=McAfee;i="6700,10204,11189"; a="28258128" X-IronPort-AV: E=Sophos;i="6.10,213,1719903600"; d="scan'208";a="28258128" Received: from orviesa009.jf.intel.com ([10.64.159.149]) by orvoesa107.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 08 Sep 2024 18:30:16 -0700 X-CSE-ConnectionGUID: cx70BTLlS3CxDDTgPHNKyA== X-CSE-MsgGUID: E/DBYOyRSA6Yo9pun0F7fg== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.10,213,1719903600"; d="scan'208";a="66486467" Received: from feng-clx.sh.intel.com ([10.239.159.50]) by orviesa009.jf.intel.com with ESMTP; 08 Sep 2024 18:30:11 -0700 From: Feng Tang To: Vlastimil Babka , Andrew Morton , Christoph Lameter , Pekka Enberg , David Rientjes , Joonsoo Kim , Roman Gushchin , Hyeonggon Yoo <42.hyeyoo@gmail.com>, Andrey Konovalov , Marco Elver , Shuah Khan , David Gow , Danilo Krummrich Cc: linux-mm@kvack.org, kasan-dev@googlegroups.com, linux-kernel@vger.kernel.org, Feng Tang Subject: [PATCH 3/5] mm/slub: Improve redzone check and zeroing for krealloc() Date: Mon, 9 Sep 2024 09:29:56 +0800 Message-Id: <20240909012958.913438-4-feng.tang@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240909012958.913438-1-feng.tang@intel.com> References: <20240909012958.913438-1-feng.tang@intel.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" For current krealloc(), one problem is its caller doesn't know what's the actual request size, say the object is 64 bytes kmalloc one, but the original caller may only requested 48 bytes. And when krealloc() shrinks or grows in the same object, or allocate a new bigger object, it lacks this 'original size' information to do accurate data preserving or zeroing (when __GFP_ZERO is set). And when some slub debug option is enabled, kmalloc caches do have this 'orig_size' feature. So utilize it to do more accurate data handling, as well as enforce the kmalloc-redzone sanity check. The krealloc() related code is moved from slab_common.c to slub.c for more efficient function calling. Suggested-by: Vlastimil Babka Signed-off-by: Feng Tang --- mm/slab_common.c | 84 ------------------------------------- mm/slub.c | 106 +++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 106 insertions(+), 84 deletions(-) diff --git a/mm/slab_common.c b/mm/slab_common.c index ad438ba62485..e59942fb7970 100644 --- a/mm/slab_common.c +++ b/mm/slab_common.c @@ -1297,90 +1297,6 @@ module_init(slab_proc_init); =20 #endif /* CONFIG_SLUB_DEBUG */ =20 -static __always_inline __realloc_size(2) void * -__do_krealloc(const void *p, size_t new_size, gfp_t flags) -{ - void *ret; - size_t ks; - - /* Check for double-free before calling ksize. */ - if (likely(!ZERO_OR_NULL_PTR(p))) { - if (!kasan_check_byte(p)) - return NULL; - ks =3D ksize(p); - } else - ks =3D 0; - - /* If the object still fits, repoison it precisely. */ - if (ks >=3D new_size) { - /* Zero out spare memory. */ - if (want_init_on_alloc(flags)) { - kasan_disable_current(); - memset((void *)p + new_size, 0, ks - new_size); - kasan_enable_current(); - } - - p =3D kasan_krealloc((void *)p, new_size, flags); - return (void *)p; - } - - ret =3D kmalloc_node_track_caller_noprof(new_size, flags, NUMA_NO_NODE, _= RET_IP_); - if (ret && p) { - /* Disable KASAN checks as the object's redzone is accessed. */ - kasan_disable_current(); - memcpy(ret, kasan_reset_tag(p), ks); - kasan_enable_current(); - } - - return ret; -} - -/** - * krealloc - reallocate memory. The contents will remain unchanged. - * @p: object to reallocate memory for. - * @new_size: how many bytes of memory are required. - * @flags: the type of memory to allocate. - * - * If @p is %NULL, krealloc() behaves exactly like kmalloc(). If @new_size - * is 0 and @p is not a %NULL pointer, the object pointed to is freed. - * - * If __GFP_ZERO logic is requested, callers must ensure that, starting wi= th the - * initial memory allocation, every subsequent call to this API for the sa= me - * memory allocation is flagged with __GFP_ZERO. Otherwise, it is possible= that - * __GFP_ZERO is not fully honored by this API. - * - * This is the case, since krealloc() only knows about the bucket size of = an - * allocation (but not the exact size it was allocated with) and hence - * implements the following semantics for shrinking and growing buffers wi= th - * __GFP_ZERO. - * - * new bucket - * 0 size size - * |--------|----------------| - * | keep | zero | - * - * In any case, the contents of the object pointed to are preserved up to = the - * lesser of the new and old sizes. - * - * Return: pointer to the allocated memory or %NULL in case of error - */ -void *krealloc_noprof(const void *p, size_t new_size, gfp_t flags) -{ - void *ret; - - if (unlikely(!new_size)) { - kfree(p); - return ZERO_SIZE_PTR; - } - - ret =3D __do_krealloc(p, new_size, flags); - if (ret && kasan_reset_tag(p) !=3D kasan_reset_tag(ret)) - kfree(p); - - return ret; -} -EXPORT_SYMBOL(krealloc_noprof); - /** * kfree_sensitive - Clear sensitive information in memory before freeing * @p: object to free memory of diff --git a/mm/slub.c b/mm/slub.c index 4cb3822dba08..d4c938dfb89e 100644 --- a/mm/slub.c +++ b/mm/slub.c @@ -4709,6 +4709,112 @@ void kfree(const void *object) } EXPORT_SYMBOL(kfree); =20 +static __always_inline __realloc_size(2) void * +__do_krealloc(const void *p, size_t new_size, gfp_t flags) +{ + void *ret; + size_t ks; + int orig_size =3D 0; + struct kmem_cache *s; + + /* Check for double-free before calling ksize. */ + if (likely(!ZERO_OR_NULL_PTR(p))) { + if (!kasan_check_byte(p)) + return NULL; + + s =3D virt_to_cache(p); + orig_size =3D get_orig_size(s, (void *)p); + ks =3D s->object_size; + } else + ks =3D 0; + + /* If the object doesn't fit, allocate a bigger one */ + if (new_size > ks) + goto alloc_new; + + /* Zero out spare memory. */ + if (want_init_on_alloc(flags)) { + kasan_disable_current(); + if (orig_size < new_size) + memset((void *)p + orig_size, 0, new_size - orig_size); + else + memset((void *)p + new_size, 0, ks - new_size); + kasan_enable_current(); + } + + if (slub_debug_orig_size(s) && !is_kfence_address(p)) { + set_orig_size(s, (void *)p, new_size); + if (s->flags & SLAB_RED_ZONE && new_size < ks) + memset_no_sanitize_memory((void *)p + new_size, + SLUB_RED_ACTIVE, ks - new_size); + } + + p =3D kasan_krealloc((void *)p, new_size, flags); + return (void *)p; + +alloc_new: + ret =3D kmalloc_node_track_caller_noprof(new_size, flags, NUMA_NO_NODE, _= RET_IP_); + if (ret && p) { + /* Disable KASAN checks as the object's redzone is accessed. */ + kasan_disable_current(); + if (orig_size) + memcpy(ret, kasan_reset_tag(p), orig_size); + kasan_enable_current(); + } + + return ret; +} + +/** + * krealloc - reallocate memory. The contents will remain unchanged. + * @p: object to reallocate memory for. + * @new_size: how many bytes of memory are required. + * @flags: the type of memory to allocate. + * + * If @p is %NULL, krealloc() behaves exactly like kmalloc(). If @new_size + * is 0 and @p is not a %NULL pointer, the object pointed to is freed. + * + * If __GFP_ZERO logic is requested, callers must ensure that, starting wi= th the + * initial memory allocation, every subsequent call to this API for the sa= me + * memory allocation is flagged with __GFP_ZERO. Otherwise, it is possible= that + * __GFP_ZERO is not fully honored by this API. + * + * When slub_debug_orig_size() is off, since krealloc() only knows about = the + * bucket size of an allocation (but not the exact size it was allocated w= ith) + * and hence implements the following semantics for shrinking and growing + * buffers with __GFP_ZERO. + * + * new bucket + * 0 size size + * |--------|----------------| + * | keep | zero | + * + * Otherwize, the original allocation size 'orig_size' could be used to + * precisely clear the requested size, and the new size will also be store= d as + * the new 'orig_size'. + * + * In any case, the contents of the object pointed to are preserved up to = the + * lesser of the new and old sizes. + * + * Return: pointer to the allocated memory or %NULL in case of error + */ +void *krealloc_noprof(const void *p, size_t new_size, gfp_t flags) +{ + void *ret; + + if (unlikely(!new_size)) { + kfree(p); + return ZERO_SIZE_PTR; + } + + ret =3D __do_krealloc(p, new_size, flags); + if (ret && kasan_reset_tag(p) !=3D kasan_reset_tag(ret)) + kfree(p); + + return ret; +} +EXPORT_SYMBOL(krealloc_noprof); + struct detached_freelist { struct slab *slab; void *tail; --=20 2.34.1 From nobody Sat Nov 30 16:42:11 2024 Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.15]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 1C0D71C286 for ; Mon, 9 Sep 2024 01:30:20 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=198.175.65.15 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1725845421; cv=none; b=qVB9ClYxc3utKK5kCrMJ4ITofKVjSSBCyj9VXbo2jsbseWhuaGCHDQZvOAbzYOXPBKkmM/sSNRIRNDgV+YyO2PWSg8oJntutr7NKaOpe4tPecVcOYqwebhPJAVo5ATe2DGeB0zQ9QaZDvW/Sj+c+D7Le++nSCnuKrx/+3/th45g= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1725845421; c=relaxed/simple; bh=sd4pFelEU/QdXj5ouoP5PRXHKn8qGUrWc/AAyu1UHpU=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=g2ZJqJIvBuBLwzFgl0p0dG6RNrzqfzlBMQwr16i5jkTJt+evyrUWEwU0SC+Xf0dE/fy5Fs8wQJkB9I1TyHzBqkHHTo17Vnvf1FkHJLm1tCWzXTxk2gZzKIgNgcZUPnfdUQTb3Xdhi/7+EOhx3UWdOztW1lmVvB7I6lbRvJ9wS+0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=Nq8Qelke; arc=none smtp.client-ip=198.175.65.15 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="Nq8Qelke" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1725845420; x=1757381420; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=sd4pFelEU/QdXj5ouoP5PRXHKn8qGUrWc/AAyu1UHpU=; b=Nq8QelkeHuFcOZnhF3FEEfIcgM3sT9yQzoPvF78Ttfv1mUx8L2ePEpaE mogHTkDHJiDiDlvzRNgiJgmHanq44ZYHwn6LfaUlpsvjkfiLir7IOm4Yc Yd6ZT8puYmkz5ncNvv6eczcwyb2CgQb26GKhP5lWh1S3KQSEbyBnMcXul J7epsH/nv5iQrYgvFIMOuV07YOJUsHIZxYR9PN9uK8OnEVTvdQLoqvjSL R28PGRTZfJzgRwHbR0W7yLEJ/vOOArs98jwu2xMGCZ1720H6b0zp5MwLe s+zM57l/KiRDhUXnu8SvK68vuCC4m7j3MOLLsUxkM6z4eFz+sdltKIreZ w==; X-CSE-ConnectionGUID: dFMCxl44SOi1HZzIx4/bug== X-CSE-MsgGUID: TkPARlyZQtSuE/G9tVquNQ== X-IronPort-AV: E=McAfee;i="6700,10204,11189"; a="28258145" X-IronPort-AV: E=Sophos;i="6.10,213,1719903600"; d="scan'208";a="28258145" Received: from orviesa009.jf.intel.com ([10.64.159.149]) by orvoesa107.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 08 Sep 2024 18:30:20 -0700 X-CSE-ConnectionGUID: WDU907LnRC+3vfLghWzWkw== X-CSE-MsgGUID: 4FEk/tfqQaKVE8AxVIBYaQ== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.10,213,1719903600"; d="scan'208";a="66486499" Received: from feng-clx.sh.intel.com ([10.239.159.50]) by orviesa009.jf.intel.com with ESMTP; 08 Sep 2024 18:30:15 -0700 From: Feng Tang To: Vlastimil Babka , Andrew Morton , Christoph Lameter , Pekka Enberg , David Rientjes , Joonsoo Kim , Roman Gushchin , Hyeonggon Yoo <42.hyeyoo@gmail.com>, Andrey Konovalov , Marco Elver , Shuah Khan , David Gow , Danilo Krummrich Cc: linux-mm@kvack.org, kasan-dev@googlegroups.com, linux-kernel@vger.kernel.org, Feng Tang Subject: [PATCH 4/5] kunit: kfence: Make KFENCE_TEST_REQUIRES macro available for all kunit case Date: Mon, 9 Sep 2024 09:29:57 +0800 Message-Id: <20240909012958.913438-5-feng.tang@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240909012958.913438-1-feng.tang@intel.com> References: <20240909012958.913438-1-feng.tang@intel.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" KFENCE_TEST_REQUIRES macro is convenient for judging if a prerequisite of a test case exists. Lift it into kunit/test.h so that all kunit test cases can benefit from it. Signed-off-by: Feng Tang Reviewed-by: Alexander Potapenko Reviewed-by: Marco Elver --- include/kunit/test.h | 6 ++++++ mm/kfence/kfence_test.c | 9 ++------- 2 files changed, 8 insertions(+), 7 deletions(-) diff --git a/include/kunit/test.h b/include/kunit/test.h index 5ac237c949a0..8a8027e10b89 100644 --- a/include/kunit/test.h +++ b/include/kunit/test.h @@ -643,6 +643,12 @@ void __printf(2, 3) kunit_log_append(struct string_str= eam *log, const char *fmt, WRITE_ONCE(test->last_seen.line, __LINE__); \ } while (0) =20 +#define KUNIT_TEST_REQUIRES(test, cond) do { \ + if (!(cond)) \ + kunit_skip((test), "Test requires: " #cond); \ +} while (0) + + /** * KUNIT_SUCCEED() - A no-op expectation. Only exists for code clarity. * @test: The test context object. diff --git a/mm/kfence/kfence_test.c b/mm/kfence/kfence_test.c index 00fd17285285..5dbb22c8c44f 100644 --- a/mm/kfence/kfence_test.c +++ b/mm/kfence/kfence_test.c @@ -32,11 +32,6 @@ #define arch_kfence_test_address(addr) (addr) #endif =20 -#define KFENCE_TEST_REQUIRES(test, cond) do { \ - if (!(cond)) \ - kunit_skip((test), "Test requires: " #cond); \ -} while (0) - /* Report as observed from console. */ static struct { spinlock_t lock; @@ -561,7 +556,7 @@ static void test_init_on_free(struct kunit *test) }; int i; =20 - KFENCE_TEST_REQUIRES(test, IS_ENABLED(CONFIG_INIT_ON_FREE_DEFAULT_ON)); + KUNIT_TEST_REQUIRES(test, IS_ENABLED(CONFIG_INIT_ON_FREE_DEFAULT_ON)); /* Assume it hasn't been disabled on command line. */ =20 setup_test_cache(test, size, 0, NULL); @@ -609,7 +604,7 @@ static void test_gfpzero(struct kunit *test) int i; =20 /* Skip if we think it'd take too long. */ - KFENCE_TEST_REQUIRES(test, kfence_sample_interval <=3D 100); + KUNIT_TEST_REQUIRES(test, kfence_sample_interval <=3D 100); =20 setup_test_cache(test, size, 0, NULL); buf1 =3D test_alloc(test, size, GFP_KERNEL, ALLOCATE_ANY); --=20 2.34.1 From nobody Sat Nov 30 16:42:11 2024 Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.15]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 06DEA1CF9A for ; Mon, 9 Sep 2024 01:30:24 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=198.175.65.15 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1725845425; cv=none; b=E4gW+6sjRC1T5QdeqhNqVQFu+l6I3gtM06uq6R8SE7Z60yExmAzuWYtyYKAA1NP6xCqW0owcPEBiTbHyFg4TQnNfC3OLwEd5F6VSp8H2jhwkBZhENHwxCyfVloUjYcV3aMhspt6oQA6Q7lqlc/K5AxiboGS90pJegCzcgCHlc18= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1725845425; c=relaxed/simple; bh=Ol5TUqWkU7uvIyG026mGuFtpsR/rEt0g9LWcDZDc0MM=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=LdK0Dr7+wgBgT1scDvl8s9fHoIarc0MJkiDtxbPhL0x5Jb2JwfCR0UAuguZ4bS06LP5rlknwf1eHlplCtUvB4sepw68/9GyNDJZsn0eBAFvY9S+zOXEpopPqYdoVnm8ZLeMScdXZAPPf9UAVd2/AQ7ooRKP1nSdJePc06/KO19Y= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=MLiaaWnj; arc=none smtp.client-ip=198.175.65.15 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="MLiaaWnj" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1725845424; x=1757381424; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=Ol5TUqWkU7uvIyG026mGuFtpsR/rEt0g9LWcDZDc0MM=; b=MLiaaWnjz/qlMjstOwedT7w+GHAVqaxijuYmEZFoZ6NTFqpRG3Ux7alx +QlOJv7lWICUPNMowS43ZEY2gYQREob2TGAUyKjmzH/K4m5yo0YFq/EXR BvvKEcTcHm53x3CNZheSzcLYEw9/5BqnDu7qtB4KXGrQB/sZCGvNRVW3b B0+KJ+jJE67GWCPOGRZ7MycyFFH55cI82Csw+HoY50t2M5qtBlc6V29JT sugZakY7Qwo+iUIJh7eXWubLHjWod8w4qPXcBhQfHhsC9w2KbUIJdDEEz yD/HhVXSIubM6f9SvShySuQ+f1Ri6e80izT0finuReRIFANcnuJ2Lbxno Q==; X-CSE-ConnectionGUID: rRtHkNVER7aFJL+IF6PMmw== X-CSE-MsgGUID: 1zuBpFx/QGmifuMkK9BKHw== X-IronPort-AV: E=McAfee;i="6700,10204,11189"; a="28258160" X-IronPort-AV: E=Sophos;i="6.10,213,1719903600"; d="scan'208";a="28258160" Received: from orviesa009.jf.intel.com ([10.64.159.149]) by orvoesa107.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 08 Sep 2024 18:30:24 -0700 X-CSE-ConnectionGUID: ROdRq+r4T9WF2KAQcT/Xlg== X-CSE-MsgGUID: DV/YFBNgQQa1SpiY83zezA== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.10,213,1719903600"; d="scan'208";a="66486541" Received: from feng-clx.sh.intel.com ([10.239.159.50]) by orviesa009.jf.intel.com with ESMTP; 08 Sep 2024 18:30:19 -0700 From: Feng Tang To: Vlastimil Babka , Andrew Morton , Christoph Lameter , Pekka Enberg , David Rientjes , Joonsoo Kim , Roman Gushchin , Hyeonggon Yoo <42.hyeyoo@gmail.com>, Andrey Konovalov , Marco Elver , Shuah Khan , David Gow , Danilo Krummrich Cc: linux-mm@kvack.org, kasan-dev@googlegroups.com, linux-kernel@vger.kernel.org, Feng Tang Subject: [PATCH 5/5] mm/slub, kunit: Add testcase for krealloc redzone and zeroing Date: Mon, 9 Sep 2024 09:29:58 +0800 Message-Id: <20240909012958.913438-6-feng.tang@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240909012958.913438-1-feng.tang@intel.com> References: <20240909012958.913438-1-feng.tang@intel.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Danilo Krummrich raised issue about krealloc+GFP_ZERO [1], and Vlastimil suggested to add some test case which can sanity test the kmalloc-redzone and zeroing by utilizing the kmalloc's 'orig_size' debug feature. It covers the grow and shrink case of krealloc() re-using current kmalloc object, and the case of re-allocating a new bigger object. User can add "slub_debug" kernel cmdline parameter to test it. [1]. https://lore.kernel.org/lkml/20240812223707.32049-1-dakr@kernel.org/ Suggested-by: Vlastimil Babka Signed-off-by: Feng Tang Reviewed-by: Danilo Krummrich --- lib/slub_kunit.c | 46 ++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 46 insertions(+) diff --git a/lib/slub_kunit.c b/lib/slub_kunit.c index 6e3a1e5a7142..03e0089149ad 100644 --- a/lib/slub_kunit.c +++ b/lib/slub_kunit.c @@ -186,6 +186,51 @@ static void test_leak_destroy(struct kunit *test) KUNIT_EXPECT_EQ(test, 1, slab_errors); } =20 +static void test_krealloc_redzone_zeroing(struct kunit *test) +{ + char *p; + int i; + + KUNIT_TEST_REQUIRES(test, __slub_debug_enabled()); + + /* Allocate a 64B kmalloc object */ + p =3D kzalloc(48, GFP_KERNEL); + if (unlikely(is_kfence_address(p))) { + kfree(p); + return; + } + memset(p, 0xff, 48); + + kasan_disable_current(); + OPTIMIZER_HIDE_VAR(p); + + /* Test shrink */ + p =3D krealloc(p, 40, GFP_KERNEL | __GFP_ZERO); + for (i =3D 40; i < 64; i++) + KUNIT_EXPECT_EQ(test, p[i], SLUB_RED_ACTIVE); + + /* Test grow within the same 64B kmalloc object */ + p =3D krealloc(p, 56, GFP_KERNEL | __GFP_ZERO); + for (i =3D 40; i < 56; i++) + KUNIT_EXPECT_EQ(test, p[i], 0); + for (i =3D 56; i < 64; i++) + KUNIT_EXPECT_EQ(test, p[i], SLUB_RED_ACTIVE); + + /* Test grow with allocating a bigger 128B object */ + p =3D krealloc(p, 112, GFP_KERNEL | __GFP_ZERO); + if (unlikely(is_kfence_address(p))) + goto exit; + + for (i =3D 56; i < 112; i++) + KUNIT_EXPECT_EQ(test, p[i], 0); + for (i =3D 112; i < 128; i++) + KUNIT_EXPECT_EQ(test, p[i], SLUB_RED_ACTIVE); + +exit: + kfree(p); + kasan_enable_current(); +} + static int test_init(struct kunit *test) { slab_errors =3D 0; @@ -196,6 +241,7 @@ static int test_init(struct kunit *test) } =20 static struct kunit_case test_cases[] =3D { + KUNIT_CASE(test_krealloc_redzone_zeroing), KUNIT_CASE(test_clobber_zone), =20 #ifndef CONFIG_KASAN --=20 2.34.1