From nobody Sun Apr 19 12:24:05 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id E9BB6C761AF for ; Mon, 3 Apr 2023 22:03:50 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233661AbjDCWDt (ORCPT ); Mon, 3 Apr 2023 18:03:49 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:56982 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232029AbjDCWDm (ORCPT ); Mon, 3 Apr 2023 18:03:42 -0400 Received: from mail-pl1-x649.google.com (mail-pl1-x649.google.com [IPv6:2607:f8b0:4864:20::649]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 99D03270F for ; Mon, 3 Apr 2023 15:03:41 -0700 (PDT) Received: by mail-pl1-x649.google.com with SMTP id p9-20020a170902e74900b001a1c7b2e7afso18011386plf.0 for ; Mon, 03 Apr 2023 15:03:41 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; t=1680559421; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=kId3QKjERA5KglVeO1VKvGvv5azlGeISt3c2gJrYgyo=; b=W/ze/sAlU7O+7GhoVMA+gO+b3zK1GLSBb2wxG7cqOl+9vjmm6QdV40GnYhvFBAr6we 6QpU1r1iDXHFCTCDI3KvZCA5Fdx66xMvZLtqAf+RggvSzScbqU6Qb+D3U4BXwhb/OMA1 mj7EUoKa/2hKudyISX9IDUT5QtGjglqN6doUhxmfcL1+w5LR9gcm9RCdAs4iQ3yY+fKw 8EP2M2Ss5aeWYRlPCW8BW8qsios85nr5jm7RooCKJYkRU/xUY3Tcdg08l4PT4At3QjPp 8RJrh+/qwzB7v9VlZ60Az/txlakHCjAKDKadt2/B61f8k1qeYBs4X+ujruMC4SFwxMjN IvmA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1680559421; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=kId3QKjERA5KglVeO1VKvGvv5azlGeISt3c2gJrYgyo=; b=JZm+cP9rP79FBj0yIznPWl6LmmSiTI/fNFMRxHEQdnI1ZHejhfGinWyyzNghPakpDi 8DZ7zOQ69wN/fsHe/Hou5D1S6LINbjKg9ocF0ylPBoN2o5SADcz2CwsGk7RKvUdoasxI 7BkHHBbSMVtOuNNanpQQ6/L0M4afFWow8HDodUPSRgL0XW7nM+3y38jW++nnauoXiUoE QiCmGMkf4lJ3NQCBJlOi2tbBk5GV3TZjr7iuinFip2zVvjjeejiYXnjY2RbK67Q/PCV/ uiTEf2i9t7h32Vo/iT2MaEv5CtfaAllyHZ3Ve60wNAwPPQLR3xJIGBrU962M96jA2qhB Y/wQ== X-Gm-Message-State: AAQBX9dZrVdlKUxTzPKNVOGHOKp+zImpTEe/XqjQT1itObWQEGDGK7wb U10Iuy/Er8dyvnC/XWYAIgBpOZNOiDIPMU06 X-Google-Smtp-Source: AKy350bROYdy/7K8uGxBM0X/ZxH2AG6/5IpCycN9GV2HdX4MbVmRhF2WLCfVj205eUvkzCs7+3h4yAAbVuROFoGd X-Received: from yosry.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:2327]) (user=yosryahmed job=sendgmr) by 2002:a63:f307:0:b0:50c:bd0:eb8c with SMTP id l7-20020a63f307000000b0050c0bd0eb8cmr39435pgh.6.1680559421112; Mon, 03 Apr 2023 15:03:41 -0700 (PDT) Date: Mon, 3 Apr 2023 22:03:33 +0000 In-Reply-To: <20230403220337.443510-1-yosryahmed@google.com> Mime-Version: 1.0 References: <20230403220337.443510-1-yosryahmed@google.com> X-Mailer: git-send-email 2.40.0.348.gf938b09366-goog Message-ID: <20230403220337.443510-2-yosryahmed@google.com> Subject: [PATCH mm-unstable RFC 1/5] writeback: move wb_over_bg_thresh() call outside lock section From: Yosry Ahmed To: Alexander Viro , Christian Brauner , Johannes Weiner , Michal Hocko , Roman Gushchin , Shakeel Butt , Muchun Song , Andrew Morton Cc: linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org, cgroups@vger.kernel.org, linux-mm@kvack.org, Yosry Ahmed Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" wb_over_bg_thresh() calls mem_cgroup_wb_stats() which invokes an rstat flush, which can be expensive on large systems. Currently, wb_writeback() calls wb_over_bg_thresh() within a lock section, so we have to make the rstat flush atomically. On systems with a lot of cpus/cgroups, this can cause us to disable irqs for a long time, potentially causing problems. Move the call to wb_over_bg_thresh() outside the lock section in preparation to make the rstat flush in mem_cgroup_wb_stats() non-atomic. The list_empty(&wb->work_list) should be okay outside the lock section of wb->list_lock as it is protected by a separate lock (wb->work_lock), and wb_over_bg_thresh() doesn't seem like it is modifying any of the b_* lists the wb->list_lock is protecting. Also, the loop seems to be already releasing and reacquring the lock, so this refactoring looks safe. Signed-off-by: Yosry Ahmed Reviewed-by: Jan Kara Reviewed-by: Michal Koutn=C3=BD --- fs/fs-writeback.c | 16 +++++++++++----- 1 file changed, 11 insertions(+), 5 deletions(-) diff --git a/fs/fs-writeback.c b/fs/fs-writeback.c index 195dc23e0d831..012357bc8daa3 100644 --- a/fs/fs-writeback.c +++ b/fs/fs-writeback.c @@ -2021,7 +2021,6 @@ static long wb_writeback(struct bdi_writeback *wb, struct blk_plug plug; =20 blk_start_plug(&plug); - spin_lock(&wb->list_lock); for (;;) { /* * Stop writeback when nr_pages has been consumed @@ -2046,6 +2045,9 @@ static long wb_writeback(struct bdi_writeback *wb, if (work->for_background && !wb_over_bg_thresh(wb)) break; =20 + + spin_lock(&wb->list_lock); + /* * Kupdate and background works are special and we want to * include all inodes that need writing. Livelock avoidance is @@ -2075,13 +2077,19 @@ static long wb_writeback(struct bdi_writeback *wb, * mean the overall work is done. So we keep looping as long * as made some progress on cleaning pages or inodes. */ - if (progress) + if (progress) { + spin_unlock(&wb->list_lock); continue; + } + /* * No more inodes for IO, bail */ - if (list_empty(&wb->b_more_io)) + if (list_empty(&wb->b_more_io)) { + spin_unlock(&wb->list_lock); break; + } + /* * Nothing written. Wait for some inode to * become available for writeback. Otherwise @@ -2093,9 +2101,7 @@ static long wb_writeback(struct bdi_writeback *wb, spin_unlock(&wb->list_lock); /* This function drops i_lock... */ inode_sleep_on_writeback(inode); - spin_lock(&wb->list_lock); } - spin_unlock(&wb->list_lock); blk_finish_plug(&plug); =20 return nr_pages - work->nr_pages; --=20 2.40.0.348.gf938b09366-goog From nobody Sun Apr 19 12:24:05 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7F31BC76188 for ; Mon, 3 Apr 2023 22:03:53 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233688AbjDCWDv (ORCPT ); Mon, 3 Apr 2023 18:03:51 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:56982 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233609AbjDCWDo (ORCPT ); Mon, 3 Apr 2023 18:03:44 -0400 Received: from mail-yw1-x1149.google.com (mail-yw1-x1149.google.com [IPv6:2607:f8b0:4864:20::1149]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 975261BF3 for ; Mon, 3 Apr 2023 15:03:43 -0700 (PDT) Received: by mail-yw1-x1149.google.com with SMTP id 00721157ae682-5446a91c40cso305705697b3.18 for ; Mon, 03 Apr 2023 15:03:43 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; t=1680559423; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=9Hf88QAEgRalaH3f7thcio7vExcVqqDxAYa/aCaE9Jc=; b=Qd1B7SbDQyGizezUFQyV/0tO/bOhj44AtZ1VKHtgOG9TbJavAnnMGZmO6zQIpqIoK7 5cj24hiW5m5RcceJDQ1KEplDfhRxIe4M8gmfj3L+NZAuLHvyis3MtYjzW6MZhBerb1QT JbsSvLpI6syO5GW80nsYNqGOYj2kCCLCJxDrHY++EHRBCuORnOG3ZRIIIY9qF02f9mrK 7QKQBjFz8zSpKFSQE5ME0/wr0Ah4Zv+Tdpoq6l9nLH2lZJPiIROIUeme5UI6HVL7rK6b PMhT9C0HP1QYba587rjt9xa6OsG8uX3e11ytS9ZKFnspyAhVnb6ce2SHS6u9IWMy0H74 szug== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1680559423; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=9Hf88QAEgRalaH3f7thcio7vExcVqqDxAYa/aCaE9Jc=; b=3DulaFOOceirPGK7isgrgoRwFREToXZL4+w0qsYE0Zv5YDjWC+A1ANPfxfPr71QFIz OicO5fmMK7jNHzgehbOlddLK+Wxf1o9PLeiwtmVbkW8X6vdZEX02WooqStsX4EFlrbNo ayIOhV2WqI67iGzS4V8ojZE4nfWrfFdUGQIdPvFVknBLxuglR0M/R6ZqSzi/yGNHSZW4 jPk2K2hbzFLmCTFe5u8wvlcxVIgvUmqu/vNfjEYo0BSa/jC9KM35+NVunmJ+4UCIq+WP cyr1Q/zqVuOvc+YXrVHhsHTvhqO9EN3t546MNj5p8TlonjYe7p9me01LhNJAtu6nPloR zaSQ== X-Gm-Message-State: AAQBX9ehX78i5t+yBQAcM1zq6lkHU2R79AuLyK0X5lEfGJV2yI81CVEZ +LY5o/RmZDANCYIK+g/CU50NMhNdCPHXtQT4 X-Google-Smtp-Source: AKy350Yk7PN55oyTS/2Ex0GZoWZBdH4qO4jUdn7c2dJd2J2mTTnpo7+qXB1LBcPp06lmU8+Do/T+aXIn4r5a27Uq X-Received: from yosry.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:2327]) (user=yosryahmed job=sendgmr) by 2002:a81:b60f:0:b0:545:bade:c57e with SMTP id u15-20020a81b60f000000b00545badec57emr288637ywh.5.1680559422857; Mon, 03 Apr 2023 15:03:42 -0700 (PDT) Date: Mon, 3 Apr 2023 22:03:34 +0000 In-Reply-To: <20230403220337.443510-1-yosryahmed@google.com> Mime-Version: 1.0 References: <20230403220337.443510-1-yosryahmed@google.com> X-Mailer: git-send-email 2.40.0.348.gf938b09366-goog Message-ID: <20230403220337.443510-3-yosryahmed@google.com> Subject: [PATCH mm-unstable RFC 2/5] memcg: flush stats non-atomically in mem_cgroup_wb_stats() From: Yosry Ahmed To: Alexander Viro , Christian Brauner , Johannes Weiner , Michal Hocko , Roman Gushchin , Shakeel Butt , Muchun Song , Andrew Morton Cc: linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org, cgroups@vger.kernel.org, linux-mm@kvack.org, Yosry Ahmed Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" The previous patch moved the wb_over_bg_thresh()->mem_cgroup_wb_stats() code path in wb_writeback() outside the lock section. We no longer need to flush the stats atomically. Flush the stats non-atomically. Signed-off-by: Yosry Ahmed Acked-by: Shakeel Butt Reviewed-by: Michal Koutn=C3=BD --- mm/memcontrol.c | 6 +----- 1 file changed, 1 insertion(+), 5 deletions(-) diff --git a/mm/memcontrol.c b/mm/memcontrol.c index 3d040a5fa7a35..bdd52fe9e7e4b 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -4637,11 +4637,7 @@ void mem_cgroup_wb_stats(struct bdi_writeback *wb, u= nsigned long *pfilepages, struct mem_cgroup *memcg =3D mem_cgroup_from_css(wb->memcg_css); struct mem_cgroup *parent; =20 - /* - * wb_writeback() takes a spinlock and calls - * wb_over_bg_thresh()->mem_cgroup_wb_stats(). Do not sleep. - */ - mem_cgroup_flush_stats_atomic(); + mem_cgroup_flush_stats(); =20 *pdirty =3D memcg_page_state(memcg, NR_FILE_DIRTY); *pwriteback =3D memcg_page_state(memcg, NR_WRITEBACK); --=20 2.40.0.348.gf938b09366-goog From nobody Sun Apr 19 12:24:05 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id C5F46C761AF for ; Mon, 3 Apr 2023 22:04:10 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233746AbjDCWEJ (ORCPT ); Mon, 3 Apr 2023 18:04:09 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:57086 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233615AbjDCWDq (ORCPT ); Mon, 3 Apr 2023 18:03:46 -0400 Received: from mail-pg1-x549.google.com (mail-pg1-x549.google.com [IPv6:2607:f8b0:4864:20::549]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 2F98335A9 for ; Mon, 3 Apr 2023 15:03:45 -0700 (PDT) Received: by mail-pg1-x549.google.com with SMTP id q15-20020a63d60f000000b00502e1c551aaso9005648pgg.21 for ; Mon, 03 Apr 2023 15:03:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; t=1680559424; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=kF9j9xhVt+HAtgUv/sW6MmUPAVRA6NHiHn3mXAJRGAA=; b=QZZrc3A9Ala3Y1fUrUeH9aHQEpEHIm56zSkCNE0A0d2CfnkVvyEn83miymwgAbVi5v 9d0zVCVjgbHefP1pXO+Nwo1Ul1PtmNl2tpG2LpPChio/My4WZElf1o1jchSNOgvJZUsW WbV5lrmHTcH3fAqoJ7gZenJSRA3/dWbn3kNYUTeryGOWXdDf83/N+i1IFVKcpHglHDCj nH4nWeMmhgItD/X0NlX5brqxxgrkgNj//KowEl9SOf+dOa58iks3E9glke7V87blHVFo T2x7kiC5rzRzeUUzMZ/tXlij1MqA5Wi+XXZnHplS+57XrL+yaVLwig14jPX/om3gIc+M IAag== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1680559424; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=kF9j9xhVt+HAtgUv/sW6MmUPAVRA6NHiHn3mXAJRGAA=; b=JWtFcwwgI+rnKFHkD62RC2ZavEcq8ZhXWDt1rurNg+y/TRMCamP0Ul4ynSVoKNeYxH V9GXqDDPOByl2i8oX2jEbxYmCwspIysZ0gco+3/JPzbhpXYTF4WKrXGfkuSkPF2uViMD PR5rSDe2cbXTnbzzHrDF6LMYlseJ1BVODucr7ZRGew5pFmpNdvuG3mklHcT3DREn/PSj FWvVC3CF/5s214gvK8N0q3xDGvL5bOyUuCmr8p5QNhADzm0O2Z0+QOVF3oJcHNZHWv/Y ya+KrVxzB9PWl43FXJGHBd2+bHfHtZGurc+oEmxsJzC0ic/Zwi0NGiJvaBeqN4sPxNoY ahzA== X-Gm-Message-State: AAQBX9c2lZVG1AxG/1UwsoFp0b2MOXK8VaZxl7NoAiBnHrImdrOpxspx 5l4JsDnb3vJkqEz/SI1H8p9ukScwQlz6B/yy X-Google-Smtp-Source: AKy350Y61C08ugQDQ0T+NdoZquMxt4ZL2sOiJByhoRsOjzxwb2aWtKxGtikw0lJtgE93Bg+tt+KjbLE8KoMgZHnh X-Received: from yosry.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:2327]) (user=yosryahmed job=sendgmr) by 2002:a63:2301:0:b0:503:91ff:8dd8 with SMTP id j1-20020a632301000000b0050391ff8dd8mr27700pgj.4.1680559424700; Mon, 03 Apr 2023 15:03:44 -0700 (PDT) Date: Mon, 3 Apr 2023 22:03:35 +0000 In-Reply-To: <20230403220337.443510-1-yosryahmed@google.com> Mime-Version: 1.0 References: <20230403220337.443510-1-yosryahmed@google.com> X-Mailer: git-send-email 2.40.0.348.gf938b09366-goog Message-ID: <20230403220337.443510-4-yosryahmed@google.com> Subject: [PATCH mm-unstable RFC 3/5] memcg: calculate root usage from global state From: Yosry Ahmed To: Alexander Viro , Christian Brauner , Johannes Weiner , Michal Hocko , Roman Gushchin , Shakeel Butt , Muchun Song , Andrew Morton Cc: linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org, cgroups@vger.kernel.org, linux-mm@kvack.org, Yosry Ahmed Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Currently, we approximate the root usage by adding the memcg stats for anon, file, and conditionally swap (for memsw). To read the memcg stats we need to invoke an rstat flush. rstat flushes can be expensive, they scale with the number of cpus and cgroups on the system. mem_cgroup_usage() is called by memcg_events()->mem_cgroup_threshold() with irqs disabled, so such an expensive operation with irqs disabled can cause problems. Instead, approximate the root usage from global state. This is not 100% accurate, but the root usage has always been ill-defined anyway. Signed-off-by: Yosry Ahmed Acked-by: Shakeel Butt Reviewed-by: Michal Koutn=C3=BD --- mm/memcontrol.c | 24 +++++------------------- 1 file changed, 5 insertions(+), 19 deletions(-) diff --git a/mm/memcontrol.c b/mm/memcontrol.c index bdd52fe9e7e4b..e7fe18c0c0ef2 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -3698,27 +3698,13 @@ static unsigned long mem_cgroup_usage(struct mem_cg= roup *memcg, bool swap) =20 if (mem_cgroup_is_root(memcg)) { /* - * We can reach here from irq context through: - * uncharge_batch() - * |--memcg_check_events() - * |--mem_cgroup_threshold() - * |--__mem_cgroup_threshold() - * |--mem_cgroup_usage - * - * rstat flushing is an expensive operation that should not be - * done from irq context; use stale stats in this case. - * Arguably, usage threshold events are not reliable on the root - * memcg anyway since its usage is ill-defined. - * - * Additionally, other call paths through memcg_check_events() - * disable irqs, so make sure we are flushing stats atomically. + * Approximate root's usage from global state. This isn't + * perfect, but the root usage was always an approximation. */ - if (in_task()) - mem_cgroup_flush_stats_atomic(); - val =3D memcg_page_state(memcg, NR_FILE_PAGES) + - memcg_page_state(memcg, NR_ANON_MAPPED); + val =3D global_node_page_state(NR_FILE_PAGES) + + global_node_page_state(NR_ANON_MAPPED); if (swap) - val +=3D memcg_page_state(memcg, MEMCG_SWAP); + val +=3D total_swap_pages - get_nr_swap_pages(); } else { if (!swap) val =3D page_counter_read(&memcg->memory); --=20 2.40.0.348.gf938b09366-goog From nobody Sun Apr 19 12:24:05 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 40A44C761AF for ; Mon, 3 Apr 2023 22:04:15 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233785AbjDCWEN (ORCPT ); Mon, 3 Apr 2023 18:04:13 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:57458 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233294AbjDCWDx (ORCPT ); Mon, 3 Apr 2023 18:03:53 -0400 Received: from mail-pg1-x54a.google.com (mail-pg1-x54a.google.com [IPv6:2607:f8b0:4864:20::54a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E00DF26A3 for ; Mon, 3 Apr 2023 15:03:46 -0700 (PDT) Received: by mail-pg1-x54a.google.com with SMTP id t12-20020a65608c000000b005091ec4f2d4so9025757pgu.20 for ; Mon, 03 Apr 2023 15:03:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; t=1680559426; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=hNJBwGVVZOmg9VOOVOlV/1s5wuc6P4CJY0BTT3hY1AM=; b=PLSbfUpOYbUqIYWD0lVUspaLdyltho/9MJ34jFGgaIu6ps/MvfYgJtvBICL0/UsCg8 Eo1bG+CpaO1W/JWBORyxWnVMwYforRdWFpLcNNnnTr6WckgWqX8cpRotu3SoW3jnNsSm PizUX3QJilPLsW/5Gft8jVYazARk9Ut3n4G+JQv0SopU/bBBAHVR6m+9um87Q19Q1oES JkmqUr13ZWZzVKrKo78OL0QtsqIHc61QjUXFxRzUrqMEuTOHPCZ7yp3LqOiDIHhK80Se /pN6/hsU4YI1MwbFb/U4Ozh2+s+tmlO9zkIjBJHts6ziq2HpcgAoxfLImivvQtFY+h9I 09eg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1680559426; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=hNJBwGVVZOmg9VOOVOlV/1s5wuc6P4CJY0BTT3hY1AM=; b=molHDdatagpyIiJ+QXM63COZWvX6CwT/WA4bLd0tznij0SgeUnQuJGMUK4Dz+a3YtV Mu0dHz+Ou33xOykk8vI7HVM2RcAdDyt/c+7XDJ+0J9u17FSy1urkpqpufe53uYmuhg1F oCt7mJCXbS1g8rhfW+9iyCBA7vuSighyF37YD2f0w+nFGFTt65SQfe6Su7sGDhvidFxT PYmuytfNsNWBV1wFWIBHzBYebOYUa7oqJFu/rWNaBRYNfFueoP59wyb13bT5Op78JsMC C3MiR+b8SCaL5IFrXWFPPUaiAttWlhdTgEaNq2NvMQcEXJlXJOA/wOnO5eMeqvyr+lgt uf/w== X-Gm-Message-State: AAQBX9fjwFxQ0axO9cIAj/EmA+2qs+MQ2zMo1JGV2mTwWCN5pMq60nVL 34qWz5phCuVLbEgTr6rTFUkaomXUDK/ELaSt X-Google-Smtp-Source: AKy350YqjuJx0cPEvkhGlL8e3G8Xhx/mHcAIKY7AoMYKpy6RY5t/7e2yXmvxs0CQV10HpQGcD3F9B9OJJYiZoWXv X-Received: from yosry.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:2327]) (user=yosryahmed job=sendgmr) by 2002:a17:90b:4cce:b0:23b:5155:309f with SMTP id nd14-20020a17090b4cce00b0023b5155309fmr7558849pjb.0.1680559426408; Mon, 03 Apr 2023 15:03:46 -0700 (PDT) Date: Mon, 3 Apr 2023 22:03:36 +0000 In-Reply-To: <20230403220337.443510-1-yosryahmed@google.com> Mime-Version: 1.0 References: <20230403220337.443510-1-yosryahmed@google.com> X-Mailer: git-send-email 2.40.0.348.gf938b09366-goog Message-ID: <20230403220337.443510-5-yosryahmed@google.com> Subject: [PATCH mm-unstable RFC 4/5] memcg: remove mem_cgroup_flush_stats_atomic() From: Yosry Ahmed To: Alexander Viro , Christian Brauner , Johannes Weiner , Michal Hocko , Roman Gushchin , Shakeel Butt , Muchun Song , Andrew Morton Cc: linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org, cgroups@vger.kernel.org, linux-mm@kvack.org, Yosry Ahmed Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Previous patches removed all callers of mem_cgroup_flush_stats_atomic(). Remove the function and simplify the code. Signed-off-by: Yosry Ahmed Acked-by: Shakeel Butt --- include/linux/memcontrol.h | 5 ----- mm/memcontrol.c | 24 +++++------------------- 2 files changed, 5 insertions(+), 24 deletions(-) diff --git a/include/linux/memcontrol.h b/include/linux/memcontrol.h index 222d7370134c7..00a88cf947e14 100644 --- a/include/linux/memcontrol.h +++ b/include/linux/memcontrol.h @@ -1038,7 +1038,6 @@ static inline unsigned long lruvec_page_state_local(s= truct lruvec *lruvec, } =20 void mem_cgroup_flush_stats(void); -void mem_cgroup_flush_stats_atomic(void); void mem_cgroup_flush_stats_ratelimited(void); =20 void __mod_memcg_lruvec_state(struct lruvec *lruvec, enum node_stat_item i= dx, @@ -1537,10 +1536,6 @@ static inline void mem_cgroup_flush_stats(void) { } =20 -static inline void mem_cgroup_flush_stats_atomic(void) -{ -} - static inline void mem_cgroup_flush_stats_ratelimited(void) { } diff --git a/mm/memcontrol.c b/mm/memcontrol.c index e7fe18c0c0ef2..33339106f1d9b 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -638,7 +638,7 @@ static inline void memcg_rstat_updated(struct mem_cgrou= p *memcg, int val) } } =20 -static void do_flush_stats(bool atomic) +static void do_flush_stats(void) { /* * We always flush the entire tree, so concurrent flushers can just @@ -651,30 +651,16 @@ static void do_flush_stats(bool atomic) =20 WRITE_ONCE(flush_next_time, jiffies_64 + 2*FLUSH_TIME); =20 - if (atomic) - cgroup_rstat_flush_atomic(root_mem_cgroup->css.cgroup); - else - cgroup_rstat_flush(root_mem_cgroup->css.cgroup); + cgroup_rstat_flush(root_mem_cgroup->css.cgroup); =20 atomic_set(&stats_flush_threshold, 0); atomic_set(&stats_flush_ongoing, 0); } =20 -static bool should_flush_stats(void) -{ - return atomic_read(&stats_flush_threshold) > num_online_cpus(); -} - void mem_cgroup_flush_stats(void) { - if (should_flush_stats()) - do_flush_stats(false); -} - -void mem_cgroup_flush_stats_atomic(void) -{ - if (should_flush_stats()) - do_flush_stats(true); + if (atomic_read(&stats_flush_threshold) > num_online_cpus()) + do_flush_stats(); } =20 void mem_cgroup_flush_stats_ratelimited(void) @@ -689,7 +675,7 @@ static void flush_memcg_stats_dwork(struct work_struct = *w) * Always flush here so that flushing in latency-sensitive paths is * as cheap as possible. */ - do_flush_stats(false); + do_flush_stats(); queue_delayed_work(system_unbound_wq, &stats_flush_dwork, FLUSH_TIME); } =20 --=20 2.40.0.348.gf938b09366-goog From nobody Sun Apr 19 12:24:05 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 25F20C76196 for ; Mon, 3 Apr 2023 22:04:20 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233454AbjDCWES (ORCPT ); Mon, 3 Apr 2023 18:04:18 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:57454 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233710AbjDCWEG (ORCPT ); Mon, 3 Apr 2023 18:04:06 -0400 Received: from mail-pl1-x649.google.com (mail-pl1-x649.google.com [IPv6:2607:f8b0:4864:20::649]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 26D704C38 for ; Mon, 3 Apr 2023 15:03:49 -0700 (PDT) Received: by mail-pl1-x649.google.com with SMTP id kp6-20020a170903280600b001a2945a7fdeso7307272plb.18 for ; Mon, 03 Apr 2023 15:03:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; t=1680559428; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=QacupsFZGNVPJSGTJC6YypXeluEyL82tnZkM1VlhIdc=; b=kWaOfIm3ix3CjQ6P0LRgdpWoqBgJu3ykmdsz32NiUF8x9A1dmCU0K7B4JSzYOReEDZ FYgPQlT3SuWLrHKrP3wDopLbn9eVfq4lg7l4naylQ8tTZgKaGOhXPPjrkCrnIEo3mSYz lk/IOCkztnHJOg8hCDeaYralpjItb0pP1L9kf7xz8p5kouBsn/T4mHo837Hsw6EVM4fX wXifin0wFGMZCFLJ9fRpD2UTXhFCMfIx9NWguUNma600XyrmJZUcuoS1IkX3lwn5WJbF /q9yx8rYhgQBtZhs0hPFucZM0Rd23Z/nGcnjDiwXILvEaD94FJtXEZgMXINCRE/Es3tO KyGQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1680559428; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=QacupsFZGNVPJSGTJC6YypXeluEyL82tnZkM1VlhIdc=; b=N/1EeCqEP8sAxMV3gidKJXraGnARu385DVHj1FqTMasW+20kDigaVYyubyR9Wvc+Nc 3WU+5YVucuoeuBuJOgse68ZSoX0Y3D+pjvJDncLdUGYV4VNSqCNqAtSNpN5j/4SSyjev Yl96WO+ikhZNmNbKXDUh3fAiTFqDg4kYnRZxOb+l0wKme8ZFuOm3bEkCzTHJW8SGEauS +sR0NeeXAposWkROnVbm/Od6l+FP8E1brYXCMk6i/BggtKcwkL41BL+ClIb/YVBy73DZ m+c2sL6KvUDd2GDRqI3ROEP2zI0Snngv1d59KbDmJxh/jZChHRfkVXgElpDSMEjs0Sgl AsYA== X-Gm-Message-State: AAQBX9fZxoq+Qyr7Q88pFcwoRwcFAjY2QIZT36WrNltemL+QtXER3TMh XvfXEQd77UW0sD/hkukGUxMZGcxh9ABQ1YT/ X-Google-Smtp-Source: AKy350bOmcX4bhvMcYbhTSAl8lBQ2O0deW3xZkttmpHKSB+VEjNEWJFCZZnxKvEdtC6SV8VkwoJDDIrEQNByEpas X-Received: from yosry.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:2327]) (user=yosryahmed job=sendgmr) by 2002:a63:485d:0:b0:4f2:8281:8afb with SMTP id x29-20020a63485d000000b004f282818afbmr35913pgk.4.1680559428241; Mon, 03 Apr 2023 15:03:48 -0700 (PDT) Date: Mon, 3 Apr 2023 22:03:37 +0000 In-Reply-To: <20230403220337.443510-1-yosryahmed@google.com> Mime-Version: 1.0 References: <20230403220337.443510-1-yosryahmed@google.com> X-Mailer: git-send-email 2.40.0.348.gf938b09366-goog Message-ID: <20230403220337.443510-6-yosryahmed@google.com> Subject: [PATCH mm-unstable RFC 5/5] cgroup: remove cgroup_rstat_flush_atomic() From: Yosry Ahmed To: Alexander Viro , Christian Brauner , Johannes Weiner , Michal Hocko , Roman Gushchin , Shakeel Butt , Muchun Song , Andrew Morton Cc: linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org, cgroups@vger.kernel.org, linux-mm@kvack.org, Yosry Ahmed Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Previous patches removed the only caller of cgroup_rstat_flush_atomic(). Remove the function and simplify the code. Signed-off-by: Yosry Ahmed --- include/linux/cgroup.h | 1 - kernel/cgroup/rstat.c | 26 +++++--------------------- 2 files changed, 5 insertions(+), 22 deletions(-) diff --git a/include/linux/cgroup.h b/include/linux/cgroup.h index 885f5395fcd04..567c547cf371f 100644 --- a/include/linux/cgroup.h +++ b/include/linux/cgroup.h @@ -692,7 +692,6 @@ static inline void cgroup_path_from_kernfs_id(u64 id, c= har *buf, size_t buflen) */ void cgroup_rstat_updated(struct cgroup *cgrp, int cpu); void cgroup_rstat_flush(struct cgroup *cgrp); -void cgroup_rstat_flush_atomic(struct cgroup *cgrp); void cgroup_rstat_flush_hold(struct cgroup *cgrp); void cgroup_rstat_flush_release(void); =20 diff --git a/kernel/cgroup/rstat.c b/kernel/cgroup/rstat.c index d3252b0416b69..f9ad33f117c82 100644 --- a/kernel/cgroup/rstat.c +++ b/kernel/cgroup/rstat.c @@ -171,7 +171,7 @@ __weak noinline void bpf_rstat_flush(struct cgroup *cgr= p, __diag_pop(); =20 /* see cgroup_rstat_flush() */ -static void cgroup_rstat_flush_locked(struct cgroup *cgrp, bool may_sleep) +static void cgroup_rstat_flush_locked(struct cgroup *cgrp) __releases(&cgroup_rstat_lock) __acquires(&cgroup_rstat_lock) { int cpu; @@ -207,9 +207,8 @@ static void cgroup_rstat_flush_locked(struct cgroup *cg= rp, bool may_sleep) } raw_spin_unlock_irqrestore(cpu_lock, flags); =20 - /* if @may_sleep, play nice and yield if necessary */ - if (may_sleep && (need_resched() || - spin_needbreak(&cgroup_rstat_lock))) { + /* play nice and yield if necessary */ + if (need_resched() || spin_needbreak(&cgroup_rstat_lock)) { spin_unlock_irq(&cgroup_rstat_lock); if (!cond_resched()) cpu_relax(); @@ -236,25 +235,10 @@ __bpf_kfunc void cgroup_rstat_flush(struct cgroup *cg= rp) might_sleep(); =20 spin_lock_irq(&cgroup_rstat_lock); - cgroup_rstat_flush_locked(cgrp, true); + cgroup_rstat_flush_locked(cgrp); spin_unlock_irq(&cgroup_rstat_lock); } =20 -/** - * cgroup_rstat_flush_atomic- atomic version of cgroup_rstat_flush() - * @cgrp: target cgroup - * - * This function can be called from any context. - */ -void cgroup_rstat_flush_atomic(struct cgroup *cgrp) -{ - unsigned long flags; - - spin_lock_irqsave(&cgroup_rstat_lock, flags); - cgroup_rstat_flush_locked(cgrp, false); - spin_unlock_irqrestore(&cgroup_rstat_lock, flags); -} - /** * cgroup_rstat_flush_hold - flush stats in @cgrp's subtree and hold * @cgrp: target cgroup @@ -269,7 +253,7 @@ void cgroup_rstat_flush_hold(struct cgroup *cgrp) { might_sleep(); spin_lock_irq(&cgroup_rstat_lock); - cgroup_rstat_flush_locked(cgrp, true); + cgroup_rstat_flush_locked(cgrp); } =20 /** --=20 2.40.0.348.gf938b09366-goog