From nobody Wed Apr 8 06:29:21 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 524EEC38A2D for ; Mon, 24 Oct 2022 08:16:42 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230358AbiJXIQk (ORCPT ); Mon, 24 Oct 2022 04:16:40 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:45200 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230260AbiJXIP7 (ORCPT ); Mon, 24 Oct 2022 04:15:59 -0400 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 6B29B33A09 for ; Mon, 24 Oct 2022 01:14:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1666599292; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=AT64JPjijrBa4KylaT1nTk9WXXduaxRTtCv2wVwy4QE=; b=AUMR4H9IEcg/RzzXZM8nxu/B7wkhJjeznn89DdTZJlcfQcqZS96Yd/mqfMPZrmpCBDxC4y ma2PO5zcvYKT/8IwWamm4M6hnPh6Tlx+NmaI+0IyfEcBlbLtYS06zwxYVYfX57NKH+TK07 WZHORlxn96te9g6rnuYqld+OAM/nusc= Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-130-mSfA_UKmOI6tRH5XBi0jPw-1; Mon, 24 Oct 2022 04:14:48 -0400 X-MC-Unique: mSfA_UKmOI6tRH5XBi0jPw-1 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.rdu2.redhat.com [10.11.54.2]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id ADB4B3C10144; Mon, 24 Oct 2022 08:14:46 +0000 (UTC) Received: from MiWiFi-R3L-srv.redhat.com (ovpn-12-35.pek2.redhat.com [10.72.12.35]) by smtp.corp.redhat.com (Postfix) with ESMTP id 493A840C6F9F; Mon, 24 Oct 2022 08:14:42 +0000 (UTC) From: Baoquan He To: linux-kernel@vger.kernel.org Cc: Dennis Zhou , Tejun Heo , Christoph Lameter , Andrew Morton , linux-mm@kvack.org, Baoquan He Subject: [PATCH 1/8] mm/percpu: remove unused pcpu_map_extend_chunks Date: Mon, 24 Oct 2022 16:14:28 +0800 Message-Id: <20221024081435.204970-2-bhe@redhat.com> In-Reply-To: <20221024081435.204970-1-bhe@redhat.com> References: <20221024081435.204970-1-bhe@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.1 on 10.11.54.2 Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Since commit 40064aeca35c ("percpu: replace area map allocator with bitmap"), it is unneeded. Signed-off-by: Baoquan He Acked-by: Dennis Zhou --- mm/percpu.c | 3 --- 1 file changed, 3 deletions(-) diff --git a/mm/percpu.c b/mm/percpu.c index 27697b2429c2..26d8cd2ca323 100644 --- a/mm/percpu.c +++ b/mm/percpu.c @@ -174,9 +174,6 @@ static DEFINE_MUTEX(pcpu_alloc_mutex); /* chunk create/= destroy, [de]pop, map ext =20 struct list_head *pcpu_chunk_lists __ro_after_init; /* chunk list slots */ =20 -/* chunks which need their map areas extended, protected by pcpu_lock */ -static LIST_HEAD(pcpu_map_extend_chunks); - /* * The number of empty populated pages, protected by pcpu_lock. * The reserved chunk doesn't contribute to the count. --=20 2.34.1 From nobody Wed Apr 8 06:29:21 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6F63DC3A59D for ; Mon, 24 Oct 2022 08:16:12 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230339AbiJXIQK (ORCPT ); Mon, 24 Oct 2022 04:16:10 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:42408 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230268AbiJXIP1 (ORCPT ); Mon, 24 Oct 2022 04:15:27 -0400 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 923B83FA01 for ; Mon, 24 Oct 2022 01:14:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1666599294; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=II9JgjXmJ1/1u2CpjGXaWwKe+WW1rdQe/wCmpKMP6aM=; b=JgrF9EyjNHEJYlFJAZ32Xx6VvsLY3n8VobP5XITfLLJZmikEcJL11kJVSrtnTcT2K01il5 r06xEUWyTbtgNUBBl0QlQBsEdPCdXi9Z/9cUiB5ur5A+tzm4tIplBZ64HqcV6v95kLLZwr /zSJCa84e6AcV3FgeLOKIHg/l+N5WVg= Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-465-bJGbBwAUPae4Iz-zpeIZZw-1; Mon, 24 Oct 2022 04:14:51 -0400 X-MC-Unique: bJGbBwAUPae4Iz-zpeIZZw-1 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.rdu2.redhat.com [10.11.54.2]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id AC69F3C10147; Mon, 24 Oct 2022 08:14:50 +0000 (UTC) Received: from MiWiFi-R3L-srv.redhat.com (ovpn-12-35.pek2.redhat.com [10.72.12.35]) by smtp.corp.redhat.com (Postfix) with ESMTP id 546C940C94EB; Mon, 24 Oct 2022 08:14:47 +0000 (UTC) From: Baoquan He To: linux-kernel@vger.kernel.org Cc: Dennis Zhou , Tejun Heo , Christoph Lameter , Andrew Morton , linux-mm@kvack.org, Baoquan He Subject: [PATCH 2/8] mm/percpu: use list_first_entry_or_null in pcpu_reclaim_populated() Date: Mon, 24 Oct 2022 16:14:29 +0800 Message-Id: <20221024081435.204970-3-bhe@redhat.com> In-Reply-To: <20221024081435.204970-1-bhe@redhat.com> References: <20221024081435.204970-1-bhe@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.1 on 10.11.54.2 Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" To replace list_empty()/list_first_entry() pair to simplify code. Signed-off-by: Baoquan He Acked-by: Dennis Zhou --- mm/percpu.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/mm/percpu.c b/mm/percpu.c index 26d8cd2ca323..a3fde4ac03a4 100644 --- a/mm/percpu.c +++ b/mm/percpu.c @@ -2143,9 +2143,9 @@ static void pcpu_reclaim_populated(void) * other accessor is the free path which only returns area back to the * allocator not touching the populated bitmap. */ - while (!list_empty(&pcpu_chunk_lists[pcpu_to_depopulate_slot])) { - chunk =3D list_first_entry(&pcpu_chunk_lists[pcpu_to_depopulate_slot], - struct pcpu_chunk, list); + while (chunk =3D list_first_entry_or_null( + &pcpu_chunk_lists[pcpu_to_depopulate_slot], + struct pcpu_chunk, list)) { WARN_ON(chunk->immutable); =20 /* --=20 2.34.1 From nobody Wed Apr 8 06:29:21 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 36373C38A2D for ; Mon, 24 Oct 2022 08:16:40 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230242AbiJXIQi (ORCPT ); Mon, 24 Oct 2022 04:16:38 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41214 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230313AbiJXIP6 (ORCPT ); Mon, 24 Oct 2022 04:15:58 -0400 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A5E9F43313 for ; Mon, 24 Oct 2022 01:15:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1666599299; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=EBzRuKDFM+71l7c7xLrT1+dLyWIbjD2WVMOLNE3ePAk=; b=MnHAroaTc6RrPXwsH+QgEgbPO/IJ4eawdOfFKdlWRoCGChSg7ldSANu9OvDV1a4FRfIBMb /4RuK75ADJukCxSS94O85pzOPrWX0aYWUUOi49Z+RrVGvSS5ZNWSqSdAtSXEJYLUzwtTfb CeqvxxJjhMwyfN4tQ2L61PHS0YmxfdQ= Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-127-SP0uFmoMOeuzBev62VLccA-1; Mon, 24 Oct 2022 04:14:55 -0400 X-MC-Unique: SP0uFmoMOeuzBev62VLccA-1 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.rdu2.redhat.com [10.11.54.2]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 94E3D1C05132; Mon, 24 Oct 2022 08:14:54 +0000 (UTC) Received: from MiWiFi-R3L-srv.redhat.com (ovpn-12-35.pek2.redhat.com [10.72.12.35]) by smtp.corp.redhat.com (Postfix) with ESMTP id 5F37140C6F75; Mon, 24 Oct 2022 08:14:51 +0000 (UTC) From: Baoquan He To: linux-kernel@vger.kernel.org Cc: Dennis Zhou , Tejun Heo , Christoph Lameter , Andrew Morton , linux-mm@kvack.org, Baoquan He Subject: [PATCH 3/8] mm/percpu: Update the code comment when creating new chunk Date: Mon, 24 Oct 2022 16:14:30 +0800 Message-Id: <20221024081435.204970-4-bhe@redhat.com> In-Reply-To: <20221024081435.204970-1-bhe@redhat.com> References: <20221024081435.204970-1-bhe@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.1 on 10.11.54.2 Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" The lock pcpu_alloc_mutex taking code has been moved to the beginning of pcpu_allo() if it's non atomic allocation. So the code comment above above pcpu_create_chunk() callsite need be updated. Signed-off-by: Baoquan He Acked-by: Dennis Zhou --- mm/percpu.c | 6 +----- 1 file changed, 1 insertion(+), 5 deletions(-) diff --git a/mm/percpu.c b/mm/percpu.c index a3fde4ac03a4..a8121302a79c 100644 --- a/mm/percpu.c +++ b/mm/percpu.c @@ -1817,16 +1817,12 @@ static void __percpu *pcpu_alloc(size_t size, size_= t align, bool reserved, =20 spin_unlock_irqrestore(&pcpu_lock, flags); =20 - /* - * No space left. Create a new chunk. We don't want multiple - * tasks to create chunks simultaneously. Serialize and create iff - * there's still no empty chunk after grabbing the mutex. - */ if (is_atomic) { err =3D "atomic alloc failed, no space left"; goto fail; } =20 + /* No space left. Create a new chunk. */ if (list_empty(&pcpu_chunk_lists[pcpu_free_slot])) { chunk =3D pcpu_create_chunk(pcpu_gfp); if (!chunk) { --=20 2.34.1 From nobody Wed Apr 8 06:29:21 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id DDB95C38A2D for ; Mon, 24 Oct 2022 08:16:52 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229692AbiJXIQv (ORCPT ); Mon, 24 Oct 2022 04:16:51 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:49116 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230286AbiJXIQC (ORCPT ); Mon, 24 Oct 2022 04:16:02 -0400 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 48A3C4D800 for ; Mon, 24 Oct 2022 01:15:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1666599305; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=L+dfYrksNn4TdevHXr7sUSqhlmuSgsZD4nCKZ/IkdDk=; b=cAUvlvdr7ErAxFrnDg3jWxpv9aR9iP+rNipsO3fUy4JQ2pDQxhIEOeTtuK8mqy7ovZWVio prdJHB2NMjLhc7Y4IlKA/DzvkA1AHxUix9LM8a5y8OXBlSmZlHXuVhcaodYYY7SqcLLMuC zzzZRENNCUDRpZ4UiC23EeKBpafZhDU= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-458-6Bt9hgApOOS0VLkaDLvcJA-1; Mon, 24 Oct 2022 04:14:59 -0400 X-MC-Unique: 6Bt9hgApOOS0VLkaDLvcJA-1 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.rdu2.redhat.com [10.11.54.2]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id ED68C185A78F; Mon, 24 Oct 2022 08:14:58 +0000 (UTC) Received: from MiWiFi-R3L-srv.redhat.com (ovpn-12-35.pek2.redhat.com [10.72.12.35]) by smtp.corp.redhat.com (Postfix) with ESMTP id 5C16240C6FA0; Mon, 24 Oct 2022 08:14:54 +0000 (UTC) From: Baoquan He To: linux-kernel@vger.kernel.org Cc: Dennis Zhou , Tejun Heo , Christoph Lameter , Andrew Morton , linux-mm@kvack.org, Baoquan He Subject: [PATCH 4/8] mm/percpu: add comment to state the empty populated pages accounting Date: Mon, 24 Oct 2022 16:14:31 +0800 Message-Id: <20221024081435.204970-5-bhe@redhat.com> In-Reply-To: <20221024081435.204970-1-bhe@redhat.com> References: <20221024081435.204970-1-bhe@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.1 on 10.11.54.2 Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" When allocating an area from a chunk, pcpu_block_update_hint_alloc() is called to update chunk metadata, including chunk's and global nr_empty_pop_pages. However, if the allocation is not atomic, some blocks may not be populated with pages yet, while we still account it here. The number of pages will be subtracted with pcpu_chunk_populated() when populating pages. Adding code comment to make that more understandable. Signed-off-by: Baoquan He Acked-by: Dennis Zhou --- mm/percpu.c | 14 +++++++++++--- 1 file changed, 11 insertions(+), 3 deletions(-) diff --git a/mm/percpu.c b/mm/percpu.c index a8121302a79c..09e407338573 100644 --- a/mm/percpu.c +++ b/mm/percpu.c @@ -831,13 +831,15 @@ static void pcpu_block_update_hint_alloc(struct pcpu_= chunk *chunk, int bit_off, =20 /* * Update s_block. - * block->first_free must be updated if the allocation takes its place. - * If the allocation breaks the contig_hint, a scan is required to - * restore this hint. */ if (s_block->contig_hint =3D=3D PCPU_BITMAP_BLOCK_BITS) nr_empty_pages++; =20 + /* + * block->first_free must be updated if the allocation takes its place. + * If the allocation breaks the contig_hint, a scan is required to + * restore this hint. + */ if (s_off =3D=3D s_block->first_free) s_block->first_free =3D find_next_zero_bit( pcpu_index_alloc_map(chunk, s_index), @@ -912,6 +914,12 @@ static void pcpu_block_update_hint_alloc(struct pcpu_c= hunk *chunk, int bit_off, } } =20 + /* + * If the allocation is not atomic, some blocks may not + * be populated with pages, while we account it here. + * The number of pages will be subtracted with + * pcpu_chunk_populated() when populating pages. + */ if (nr_empty_pages) pcpu_update_empty_pages(chunk, -nr_empty_pages); =20 --=20 2.34.1 From nobody Wed Apr 8 06:29:21 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id CC305C3A59D for ; Mon, 24 Oct 2022 08:17:19 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230018AbiJXIRR (ORCPT ); Mon, 24 Oct 2022 04:17:17 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:48916 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230312AbiJXIQh (ORCPT ); Mon, 24 Oct 2022 04:16:37 -0400 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 7488E5D0E5 for ; Mon, 24 Oct 2022 01:15:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1666599315; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=lMCNuRuv+p/VyO8f2sd5NbLS+a0poTkVGW4v1nMo8cA=; b=OAPboD57Tw1rxbkMuw+eIV0yL0zgiwyjnlV8QHB36bErKTWz0F35lQku2BqYHhNgONu31Y AC8azW8mSlwIE40Ung+p5XebdtrfJTdsOj1vkPahj2zdvn8ygnZYd9DWkAIH5Eyu+sl8rU XsiAy8dGPY/Bi5ajijHvENAtUy4sJVk= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-523-oh6QCyUQPPSLXucDD99q_w-1; Mon, 24 Oct 2022 04:15:04 -0400 X-MC-Unique: oh6QCyUQPPSLXucDD99q_w-1 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.rdu2.redhat.com [10.11.54.2]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 4611181DA23; Mon, 24 Oct 2022 08:15:03 +0000 (UTC) Received: from MiWiFi-R3L-srv.redhat.com (ovpn-12-35.pek2.redhat.com [10.72.12.35]) by smtp.corp.redhat.com (Postfix) with ESMTP id C4C4640C6F9F; Mon, 24 Oct 2022 08:14:59 +0000 (UTC) From: Baoquan He To: linux-kernel@vger.kernel.org Cc: Dennis Zhou , Tejun Heo , Christoph Lameter , Andrew Morton , linux-mm@kvack.org, Baoquan He Subject: [PATCH 5/8] mm/percpu: replace the goto with break Date: Mon, 24 Oct 2022 16:14:32 +0800 Message-Id: <20221024081435.204970-6-bhe@redhat.com> In-Reply-To: <20221024081435.204970-1-bhe@redhat.com> References: <20221024081435.204970-1-bhe@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.1 on 10.11.54.2 Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" In function pcpu_reclaim_populated(), the line of goto jumping is unnecessary since the label 'end_chunk' is near the end of the for loop, use break instead. Signed-off-by: Baoquan He Acked-by: Dennis Zhou --- mm/percpu.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/mm/percpu.c b/mm/percpu.c index 09e407338573..954abf916c7d 100644 --- a/mm/percpu.c +++ b/mm/percpu.c @@ -2167,7 +2167,7 @@ static void pcpu_reclaim_populated(void) /* reintegrate chunk to prevent atomic alloc failures */ if (pcpu_nr_empty_pop_pages < PCPU_EMPTY_POP_PAGES_HIGH) { reintegrate =3D true; - goto end_chunk; + break; } =20 /* @@ -2203,7 +2203,6 @@ static void pcpu_reclaim_populated(void) end =3D -1; } =20 -end_chunk: /* batch tlb flush per chunk to amortize cost */ if (freed_page_start < freed_page_end) { spin_unlock_irq(&pcpu_lock); --=20 2.34.1 From nobody Wed Apr 8 06:29:21 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1E7E6C3A59D for ; Mon, 24 Oct 2022 08:17:13 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229665AbiJXIRK (ORCPT ); Mon, 24 Oct 2022 04:17:10 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:42104 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230026AbiJXIQ3 (ORCPT ); Mon, 24 Oct 2022 04:16:29 -0400 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 952E258169 for ; Mon, 24 Oct 2022 01:15:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1666599313; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=g3MWDQZ+YEFLhc1+KrcZwWlcxn1kI2z+4saRqDzwGGg=; b=NtxNg0iHzGYBL0/u08pCAo8H38CwXaXcI/da0n/quz/rz6rEhuC33FTjAs4WiLP9Q7uvc2 UusPO2mNUOOukg2zbny3aQRKOZAeJpW9ywdf9tTFbEb9wR3bNfsSWXfzjh4xl/k+3UrmH4 pvfAvxG48RPpnnCmoE7+1fcHtldRmf8= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-212-wHg3keIwOaOlWtfbqML-FQ-1; Mon, 24 Oct 2022 04:15:07 -0400 X-MC-Unique: wHg3keIwOaOlWtfbqML-FQ-1 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.rdu2.redhat.com [10.11.54.2]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 5A019101A56C; Mon, 24 Oct 2022 08:15:07 +0000 (UTC) Received: from MiWiFi-R3L-srv.redhat.com (ovpn-12-35.pek2.redhat.com [10.72.12.35]) by smtp.corp.redhat.com (Postfix) with ESMTP id EE37940C6FA0; Mon, 24 Oct 2022 08:15:03 +0000 (UTC) From: Baoquan He To: linux-kernel@vger.kernel.org Cc: Dennis Zhou , Tejun Heo , Christoph Lameter , Andrew Morton , linux-mm@kvack.org, Baoquan He Subject: [PATCH 6/8] mm/percpu.c: remove the lcm code since block size is fixed at page size Date: Mon, 24 Oct 2022 16:14:33 +0800 Message-Id: <20221024081435.204970-7-bhe@redhat.com> In-Reply-To: <20221024081435.204970-1-bhe@redhat.com> References: <20221024081435.204970-1-bhe@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.1 on 10.11.54.2 Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Since commit b239f7daf553 ("percpu: set PCPU_BITMAP_BLOCK_SIZE to PAGE_SIZE"), the PCPU_BITMAP_BLOCK_SIZE has been set to page size fixedly. So the lcm code in pcpu_alloc_first_chunk() doesn't make sense any more, clean it up. Signed-off-by: Baoquan He Acked-by: Dennis Zhou --- mm/percpu.c | 12 ++---------- 1 file changed, 2 insertions(+), 10 deletions(-) diff --git a/mm/percpu.c b/mm/percpu.c index 954abf916c7d..96191f7b61ba 100644 --- a/mm/percpu.c +++ b/mm/percpu.c @@ -72,7 +72,6 @@ #include #include #include -#include #include #include #include @@ -1347,7 +1346,7 @@ static struct pcpu_chunk * __init pcpu_alloc_first_ch= unk(unsigned long tmp_addr, int map_size) { struct pcpu_chunk *chunk; - unsigned long aligned_addr, lcm_align; + unsigned long aligned_addr; int start_offset, offset_bits, region_size, region_bits; size_t alloc_size; =20 @@ -1355,14 +1354,7 @@ static struct pcpu_chunk * __init pcpu_alloc_first_c= hunk(unsigned long tmp_addr, aligned_addr =3D tmp_addr & PAGE_MASK; =20 start_offset =3D tmp_addr - aligned_addr; - - /* - * Align the end of the region with the LCM of PAGE_SIZE and - * PCPU_BITMAP_BLOCK_SIZE. One of these constants is a multiple of - * the other. - */ - lcm_align =3D lcm(PAGE_SIZE, PCPU_BITMAP_BLOCK_SIZE); - region_size =3D ALIGN(start_offset + map_size, lcm_align); + region_size =3D ALIGN(start_offset + map_size, PAGE_SIZE); =20 /* allocate chunk */ alloc_size =3D struct_size(chunk, populated, --=20 2.34.1 From nobody Wed Apr 8 06:29:21 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id DD61EC38A2D for ; Mon, 24 Oct 2022 08:17:22 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229983AbiJXIRU (ORCPT ); Mon, 24 Oct 2022 04:17:20 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:45198 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230315AbiJXIQh (ORCPT ); Mon, 24 Oct 2022 04:16:37 -0400 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id CE5E85EDE0 for ; Mon, 24 Oct 2022 01:15:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1666599316; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=qhNjNie8t1+sw/Ww+09Nk0F1+B4cpGUlQfgZx8SKUOg=; b=fBbqTS4YDCDaKJjwxT6EfG/P+FLvYMvsTzpp5jGEPECZXzP7PWoo4kNrLrmJWQn7cuGtSL Q2tMnbvoRKcgs8UZa1SJUCv3zTfhoTwlf7Fii8udPLFOKSreK6QC7l0Z+xf0SXex+eesyW yxB2QWoWiQa85OmDzmk1z9TtejTzoNY= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-298--o0yKi-_O3u9wnEwYSAFNQ-1; Mon, 24 Oct 2022 04:15:12 -0400 X-MC-Unique: -o0yKi-_O3u9wnEwYSAFNQ-1 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.rdu2.redhat.com [10.11.54.2]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id BBF3285A5B6; Mon, 24 Oct 2022 08:15:11 +0000 (UTC) Received: from MiWiFi-R3L-srv.redhat.com (ovpn-12-35.pek2.redhat.com [10.72.12.35]) by smtp.corp.redhat.com (Postfix) with ESMTP id 29F9D40C6F9F; Mon, 24 Oct 2022 08:15:07 +0000 (UTC) From: Baoquan He To: linux-kernel@vger.kernel.org Cc: Dennis Zhou , Tejun Heo , Christoph Lameter , Andrew Morton , linux-mm@kvack.org, Baoquan He Subject: [PATCH 7/8] mm/percpu: remove unused PERCPU_DYNAMIC_EARLY_SLOTS Date: Mon, 24 Oct 2022 16:14:34 +0800 Message-Id: <20221024081435.204970-8-bhe@redhat.com> In-Reply-To: <20221024081435.204970-1-bhe@redhat.com> References: <20221024081435.204970-1-bhe@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.1 on 10.11.54.2 Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Since commit 40064aeca35c ("percpu: replace area map allocator with bitmap"), there's no place to use PERCPU_DYNAMIC_EARLY_SLOTS. So clean it up. Signed-off-by: Baoquan He Acked-by: Dennis Zhou --- include/linux/percpu.h | 7 +++---- 1 file changed, 3 insertions(+), 4 deletions(-) diff --git a/include/linux/percpu.h b/include/linux/percpu.h index f1ec5ad1351c..70bc17db00a6 100644 --- a/include/linux/percpu.h +++ b/include/linux/percpu.h @@ -37,11 +37,10 @@ /* * Percpu allocator can serve percpu allocations before slab is * initialized which allows slab to depend on the percpu allocator. - * The following two parameters decide how much resource to - * preallocate for this. Keep PERCPU_DYNAMIC_RESERVE equal to or - * larger than PERCPU_DYNAMIC_EARLY_SIZE. + * The following parameter decide how much resource to preallocate + * for this. Keep PERCPU_DYNAMIC_RESERVE equal to or larger than + * PERCPU_DYNAMIC_EARLY_SIZE. */ -#define PERCPU_DYNAMIC_EARLY_SLOTS 128 #define PERCPU_DYNAMIC_EARLY_SIZE (12 << 10) =20 /* --=20 2.34.1 From nobody Wed Apr 8 06:29:21 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1919BC3A59D for ; Mon, 24 Oct 2022 08:18:50 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230293AbiJXISs (ORCPT ); Mon, 24 Oct 2022 04:18:48 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50194 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229891AbiJXISM (ORCPT ); Mon, 24 Oct 2022 04:18:12 -0400 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A516E65008 for ; Mon, 24 Oct 2022 01:16:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1666599330; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=oB5FHCqdQzRcpJOgYsd1zcN5d+GwusZuRHj9Ye7P85Q=; b=MVVPlPV/zqchGBRP0L9jJ4IXX1nNCVDaaVg/AR6d6GrM6k8ReNJrJzgu59nRLD4umPcNJg Sdvk4qEZNx2AdpLs5tmLwSiaIbzCScmh/rz+B9XVM5E3adN44QTZfqoZlDkcNm6ZKgug86 L5kBwgxIK7jeeB1ZdKOPMYcpWaEpSVY= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-558-md_8dzoMOzCwdktsUExyUg-1; Mon, 24 Oct 2022 04:15:20 -0400 X-MC-Unique: md_8dzoMOzCwdktsUExyUg-1 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.rdu2.redhat.com [10.11.54.2]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 70B7E887400; Mon, 24 Oct 2022 08:15:19 +0000 (UTC) Received: from MiWiFi-R3L-srv.redhat.com (ovpn-12-35.pek2.redhat.com [10.72.12.35]) by smtp.corp.redhat.com (Postfix) with ESMTP id 8D63C40C6F9F; Mon, 24 Oct 2022 08:15:12 +0000 (UTC) From: Baoquan He To: linux-kernel@vger.kernel.org Cc: Dennis Zhou , Tejun Heo , Christoph Lameter , Andrew Morton , linux-mm@kvack.org, Baoquan He , Pekka Enberg , David Rientjes , Joonsoo Kim , Vlastimil Babka , Roman Gushchin , Hyeonggon Yoo <42.hyeyoo@gmail.com> Subject: [PATCH 8/8] mm/slub, percpu: correct the calculation of early percpu allocation size Date: Mon, 24 Oct 2022 16:14:35 +0800 Message-Id: <20221024081435.204970-9-bhe@redhat.com> In-Reply-To: <20221024081435.204970-1-bhe@redhat.com> References: <20221024081435.204970-1-bhe@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.1 on 10.11.54.2 Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" SLUB allocator relies on percpu allocator to initialize its ->cpu_slab during early boot. For that, the dynamic chunk of percpu which serves the early allocation need be large enough to satisfy the kmalloc creation. However, the current BUILD_BUG_ON() in alloc_kmem_cache_cpus() doesn't consider the kmalloc array with NR_KMALLOC_TYPES length. Fix that with correct calculation. Signed-off-by: Baoquan He Cc: Christoph Lameter Cc: Pekka Enberg Cc: David Rientjes Cc: Joonsoo Kim Cc: Andrew Morton Cc: Vlastimil Babka Cc: Roman Gushchin Cc: Hyeonggon Yoo <42.hyeyoo@gmail.com> Acked-by: Dennis Zhou Acked-by: Hyeonggon Yoo <42.hyeyoo@gmail.com> --- mm/slub.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/mm/slub.c b/mm/slub.c index 157527d7101b..8ac3bb9a122a 100644 --- a/mm/slub.c +++ b/mm/slub.c @@ -4017,7 +4017,8 @@ init_kmem_cache_node(struct kmem_cache_node *n) static inline int alloc_kmem_cache_cpus(struct kmem_cache *s) { BUILD_BUG_ON(PERCPU_DYNAMIC_EARLY_SIZE < - KMALLOC_SHIFT_HIGH * sizeof(struct kmem_cache_cpu)); + NR_KMALLOC_TYPES * KMALLOC_SHIFT_HIGH * + sizeof(struct kmem_cache_cpu)); =20 /* * Must align to double word boundary for the double cmpxchg --=20 2.34.1