From nobody Wed Sep 17 18:19:40 2025 Received: from mail-pg1-f201.google.com (mail-pg1-f201.google.com [209.85.215.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 5509723B639 for ; Mon, 21 Jul 2025 20:36:29 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.215.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1753130190; cv=none; b=ccOzB1VIZg+z66tjccdeaGRHkQuFeA0XvpCcctIAwISUdYJcrAZLjH/GwmsR0BbTCIbGswm0AACF/Q0aLmvPEQnkaQYwR08lfkYLxWYYPPkYtQJr9n7NXMZMr+VaH2cmOmVVgphhMG8QszZ3Z+ctukLW1uQ22UPUnch03LVEe4w= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1753130190; c=relaxed/simple; bh=nQzbvBzPyuVJ5Itkp/dHQSG157Y+6VU++58CwOQo3D4=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=GHMrlnijCGoy0TKo8Pp63rDldsabneSsbMTdrv+y5PCF4tectwkF8bMgnY2BQ3rympvNN0DWZ8i+S8TuFaN9WFgHiQmroPUdlrBmEaPPgJlP3tCZAg3PxFMqXsaRTcNqAvOoJU9g++32hruqGkGvPBfiLpuQ5WQEq0QGhrN8qJE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--kuniyu.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=gJ/g/TIv; arc=none smtp.client-ip=209.85.215.201 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--kuniyu.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="gJ/g/TIv" Received: by mail-pg1-f201.google.com with SMTP id 41be03b00d2f7-b3f33295703so3788651a12.1 for ; Mon, 21 Jul 2025 13:36:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1753130188; x=1753734988; darn=lists.linux.dev; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=P67Y4nPe97i+SWJPpvtux7JQJ2xYIXeLm24bM5pwnME=; b=gJ/g/TIv3NGt6dXKJMzipwMafSUEeVK/09h+xafhzgHKS0AVC16XHtftyXrCIW9NCw lrY+8JD6PaI3QANCPxvZB2pO7zNjm7oCh3IMUXMVMw1CDvw6ZK4E44RCZimfF+QegcZz r0y4ozQOvZ+ZT7SGfS02ortRpMCIkAdYgsVtBV8rP6tSJc5tJvHcS8wn6UZJqH1E55fS XUxSsegnJNJv6YkfHKbDjjr/HTGdVx2XslwpVKmEVlo8ymJRHTi6PTu/qj7qZw4She/q oo3C1UxlepqiOjv6hNU7o2XSPIx4Aubse0vpMzS4arVUNckVRf0rjNxHa0d2x6Fwl9kI Kgyg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1753130188; x=1753734988; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=P67Y4nPe97i+SWJPpvtux7JQJ2xYIXeLm24bM5pwnME=; b=hdjsHja3CL/tJoPi6o9hFREtbIihzAnbj5bP1SZElgp6z+1O2ZHnjeR5E0Fcdf+bwe rH16taerlqVADSQwzmSfMtvlfPDM3ywzLcTVt7IKBLnYNwlBUjC1BlTRbQplcCrA2osY OK31ehz3s+iFN9kUAOG1+fMjq2gdwp8dGW+/rBi0+Vw5hvXiSYJ990Q5jAXHnpXRq7mM G/HhnoGr6q3MZytpkrgTKW8hv8e/mRwxrWoVAe2FLHXPkys5e2lD2qVNidOIhY8AJRsF SB4lBGC9/w8AG7AwNmCLmQ3lKCazsr0do3o4PUF+PSecWnEKwAnp0hfikpUGcmLkZJgA 9dTA== X-Forwarded-Encrypted: i=1; AJvYcCULzShNqmYjUhrUeqpTTY2299Po3lrQxO6Kub1iJgsflDM/J/NQf+NhWdHUumFcUArYDlmgtQ==@lists.linux.dev X-Gm-Message-State: AOJu0YxO+0AV3pWzAVGy1KKU87vfWJbdU0B3dTRi+88ebYS+VYajdhId RbgsGk3FUKIHRNXZXHT8a4BftJw944HhlqQ6CrV8OoKoRHsm6++mCDX6HVlJVsde5qDwHuCUJHx IQkeHtQ== X-Google-Smtp-Source: AGHT+IGq5a7mBT9K4l6RJAVl9SuPJnfEZ/pSiUesxpLyMFOJ6g+rCdlI3WLMNPagI5cWYxYgX95XYihYFQ4= X-Received: from pfbhb8.prod.google.com ([2002:a05:6a00:8588:b0:748:55b9:ffbe]) (user=kuniyu job=prod-delivery.src-stubby-dispatcher) by 2002:a05:6a20:e613:b0:21f:4631:811c with SMTP id adf61e73a8af0-2390dba61fbmr26139263637.19.1753130188582; Mon, 21 Jul 2025 13:36:28 -0700 (PDT) Date: Mon, 21 Jul 2025 20:35:20 +0000 In-Reply-To: <20250721203624.3807041-1-kuniyu@google.com> Precedence: bulk X-Mailing-List: mptcp@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250721203624.3807041-1-kuniyu@google.com> X-Mailer: git-send-email 2.50.0.727.gbf7dc18ff4-goog Message-ID: <20250721203624.3807041-2-kuniyu@google.com> Subject: [PATCH v1 net-next 01/13] mptcp: Fix up subflow's memcg when CONFIG_SOCK_CGROUP_DATA=n. From: Kuniyuki Iwashima To: "David S. Miller" , Eric Dumazet , Jakub Kicinski , Neal Cardwell , Paolo Abeni , Willem de Bruijn , Matthieu Baerts , Mat Martineau , Johannes Weiner , Michal Hocko , Roman Gushchin , Shakeel Butt , Andrew Morton Cc: Simon Horman , Geliang Tang , Muchun Song , Kuniyuki Iwashima , Kuniyuki Iwashima , netdev@vger.kernel.org, mptcp@lists.linux.dev, cgroups@vger.kernel.org, linux-mm@kvack.org Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" When sk_alloc() allocates a socket, mem_cgroup_sk_alloc() sets sk->sk_memcg based on the current task. MPTCP subflow socket creation is triggered from userspace or an in-kernel worker. In the latter case, sk->sk_memcg is not what we want. So, we fix it up from the parent socket's sk->sk_memcg in mptcp_attach_cgroup(). Although the code is placed under #ifdef CONFIG_MEMCG, it is buried under #ifdef CONFIG_SOCK_CGROUP_DATA. The two configs are orthogonal. If CONFIG_MEMCG is enabled without CONFIG_SOCK_CGROUP_DATA, the subflow's memory usage is not charged correctly. Let's move the code out of the wrong ifdef guard. Note that sk->sk_memcg is freed in sk_prot_free() and the parent sk holds the refcnt of memcg->css here, so we don't need to use css_tryget(). Fixes: 3764b0c5651e3 ("mptcp: attach subflow socket to parent cgroup") Signed-off-by: Kuniyuki Iwashima Reviewed-by: Eric Dumazet --- include/linux/memcontrol.h | 7 +++++++ mm/memcontrol.c | 10 ++++++++++ net/mptcp/subflow.c | 11 +++-------- 3 files changed, 20 insertions(+), 8 deletions(-) diff --git a/include/linux/memcontrol.h b/include/linux/memcontrol.h index 87b6688f124a7..d8319ad5e8ea7 100644 --- a/include/linux/memcontrol.h +++ b/include/linux/memcontrol.h @@ -1602,6 +1602,8 @@ extern struct static_key_false memcg_sockets_enabled_= key; #define mem_cgroup_sockets_enabled static_branch_unlikely(&memcg_sockets_e= nabled_key) void mem_cgroup_sk_alloc(struct sock *sk); void mem_cgroup_sk_free(struct sock *sk); +void mem_cgroup_sk_inherit(const struct sock *sk, struct sock *newsk); + static inline bool mem_cgroup_under_socket_pressure(struct mem_cgroup *mem= cg) { #ifdef CONFIG_MEMCG_V1 @@ -1623,6 +1625,11 @@ void reparent_shrinker_deferred(struct mem_cgroup *m= emcg); #define mem_cgroup_sockets_enabled 0 static inline void mem_cgroup_sk_alloc(struct sock *sk) { }; static inline void mem_cgroup_sk_free(struct sock *sk) { }; + +static inline void mem_cgroup_sk_inherit(const struct sock *sk, struct soc= k *newsk) +{ +} + static inline bool mem_cgroup_under_socket_pressure(struct mem_cgroup *mem= cg) { return false; diff --git a/mm/memcontrol.c b/mm/memcontrol.c index 70fdeda1120b3..54eb25d8d555c 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -5090,6 +5090,16 @@ void mem_cgroup_sk_free(struct sock *sk) css_put(&sk->sk_memcg->css); } =20 +void mem_cgroup_sk_inherit(const struct sock *sk, struct sock *newsk) +{ + if (sk->sk_memcg =3D=3D newsk->sk_memcg) + return; + + mem_cgroup_sk_free(newsk); + css_get(&sk->sk_memcg->css); + newsk->sk_memcg =3D sk->sk_memcg; +} + /** * mem_cgroup_charge_skmem - charge socket memory * @memcg: memcg to charge diff --git a/net/mptcp/subflow.c b/net/mptcp/subflow.c index 1802bc5435a1a..f21d90fb1a19d 100644 --- a/net/mptcp/subflow.c +++ b/net/mptcp/subflow.c @@ -1716,19 +1716,14 @@ static void mptcp_attach_cgroup(struct sock *parent= , struct sock *child) /* only the additional subflows created by kworkers have to be modified */ if (cgroup_id(sock_cgroup_ptr(parent_skcd)) !=3D cgroup_id(sock_cgroup_ptr(child_skcd))) { -#ifdef CONFIG_MEMCG - struct mem_cgroup *memcg =3D parent->sk_memcg; - - mem_cgroup_sk_free(child); - if (memcg && css_tryget(&memcg->css)) - child->sk_memcg =3D memcg; -#endif /* CONFIG_MEMCG */ - cgroup_sk_free(child_skcd); *child_skcd =3D *parent_skcd; cgroup_sk_clone(child_skcd); } #endif /* CONFIG_SOCK_CGROUP_DATA */ + + if (mem_cgroup_sockets_enabled && parent->sk_memcg) + mem_cgroup_sk_inherit(parent, child); } =20 static void mptcp_subflow_ops_override(struct sock *ssk) --=20 2.50.0.727.gbf7dc18ff4-goog From nobody Wed Sep 17 18:19:40 2025 Received: from mail-pf1-f202.google.com (mail-pf1-f202.google.com [209.85.210.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C924323D2A8 for ; Mon, 21 Jul 2025 20:36:30 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1753130192; cv=none; b=G3Jgpmg+TDg6sfSazzbtap5kXl+GOBEzXzB4n3zf1XSbuRwZV8VKyIcM8w12RfuxRbKEjSecpyTrzEPDy4Dzmu9UqAJLFMpd2/iOAU93sWFaK5EBOz4N4rVu4KDuCKfnSdhIAyLFzBlT0PHHfgKRUeIYYsOTOfJnOypo+sTPR0M= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1753130192; c=relaxed/simple; bh=lqqyywnLH34ueK612ieCcIgpjbH8J04q4qB8KlcMhUE=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=qWcSg3RnRE2y049A/PMQz4feulNyAxPB6bHWBWiotC8w5AELKEjUVdQESTwYmLdyoks3dufd2u4iKe1tGQDZpyV0OIgcne0yyOOtj9WjLsJTzP3FHO8H615VJSBHVdNKibjKWJOYfCXl9oN17+4JGG3ZTr6x9caYj4m8jZV/ai8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--kuniyu.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=2oAq+ELV; arc=none smtp.client-ip=209.85.210.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--kuniyu.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="2oAq+ELV" Received: by mail-pf1-f202.google.com with SMTP id d2e1a72fcca58-748f3613e6aso2293642b3a.0 for ; Mon, 21 Jul 2025 13:36:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1753130190; x=1753734990; darn=lists.linux.dev; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=Cc1Ez/xk0myRMpWYi7JwhYCdmsN/iTRwifsr6matvKU=; b=2oAq+ELVJn8lbKUyzrRNJN6/JC52OSW7NFPUOLtOW+I9klj+LQJk4714pmTDGnctqX ClKaDz5PZkIFEp2J6P1jYLMoNiVLWCXvzSsT5f51g1go21gArM8JA9bD3e685VIoueY3 41K5cH23pKuCjZsgctDtuX584B7Qsjaeq0ZkKKbrS8cdWQX93jU0nupBZZ8o6HkMRuxr 0MZExM7xr/lTXqS2e9itIkZQ2wLF6XIm8vBtoap22gNPEMmi7Cjxo7Y92x3d71Xhx1zr mcS7Oi3hmmVOgA6hjgZ58F0L+Wyy5T1o1iwBC4omB6fCenYwPc59dLssqqu6vk+CdlnD KPtg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1753130190; x=1753734990; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=Cc1Ez/xk0myRMpWYi7JwhYCdmsN/iTRwifsr6matvKU=; b=FVXTbT1CM14yLT3dXuGNMD01nX3FnOvu8srJSg+aXCxOU4LUycBdPY30yqHJpXgwAq VVXXiLrpQzf/9HqmXaplGrduZ5nrrHTNE9vfRjRBG091E4OVV6hZerNf3ZdiKKbsAdKj g8HC7NK+KiOW74+63LDdlhSNzLA+VoreZrofunQ0xhVjoBLJOnRYMq+6P1zFfYA/4xot 1cpEQfQHVpEsXk6uPk8wt90ceebAZ73ZWrkhxmvAEqP8GWpwCSzAJ1BY4gbTn2doCjzP zxaIg8Kl5YoMkKuJgUWYwGMAATpyCNfWVjfPet7NSgHykad/ee0tO3oqzzHok2WQ9DqP jnuQ== X-Forwarded-Encrypted: i=1; AJvYcCUoQKWwtU7JiBFm17hQSEZQ8Y2QAaLDTufVSOY28xF6xp2SJN1w3Bz9vrFz9VMpnRo9pozA2A==@lists.linux.dev X-Gm-Message-State: AOJu0YzXAXX67kkgCE2SS6KjOu87TsVlxAczFD4T8iJZavPsKql/ETgL ahqSi7cFwM1jHHuWrebpghlr6eQhaYZJmTa8AbHbW9NEUHpmVVIB8iGKuxvqZW8UB7hVTtojHWh OKQtBpQ== X-Google-Smtp-Source: AGHT+IGaUy/HgUE6Lp8HLDEGr2kbd1XL7Q/OQ0eUpQ2PrdZCIeYCL6Si0ZJ9Rped6b5KZL3DccuQO15A9BE= X-Received: from pfbhd9.prod.google.com ([2002:a05:6a00:6589:b0:746:1a7b:a39a]) (user=kuniyu job=prod-delivery.src-stubby-dispatcher) by 2002:a05:6a00:3a05:b0:742:aecc:c472 with SMTP id d2e1a72fcca58-7572266ffdemr27218148b3a.2.1753130190133; Mon, 21 Jul 2025 13:36:30 -0700 (PDT) Date: Mon, 21 Jul 2025 20:35:21 +0000 In-Reply-To: <20250721203624.3807041-1-kuniyu@google.com> Precedence: bulk X-Mailing-List: mptcp@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250721203624.3807041-1-kuniyu@google.com> X-Mailer: git-send-email 2.50.0.727.gbf7dc18ff4-goog Message-ID: <20250721203624.3807041-3-kuniyu@google.com> Subject: [PATCH v1 net-next 02/13] mptcp: Use tcp_under_memory_pressure() in mptcp_epollin_ready(). From: Kuniyuki Iwashima To: "David S. Miller" , Eric Dumazet , Jakub Kicinski , Neal Cardwell , Paolo Abeni , Willem de Bruijn , Matthieu Baerts , Mat Martineau , Johannes Weiner , Michal Hocko , Roman Gushchin , Shakeel Butt , Andrew Morton Cc: Simon Horman , Geliang Tang , Muchun Song , Kuniyuki Iwashima , Kuniyuki Iwashima , netdev@vger.kernel.org, mptcp@lists.linux.dev, cgroups@vger.kernel.org, linux-mm@kvack.org Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Some conditions used in mptcp_epollin_ready() are the same as tcp_under_memory_pressure(). We will modify tcp_under_memory_pressure() in the later patch. Let's use tcp_under_memory_pressure() instead. Signed-off-by: Kuniyuki Iwashima Reviewed-by: Eric Dumazet --- net/mptcp/protocol.h | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-) diff --git a/net/mptcp/protocol.h b/net/mptcp/protocol.h index 6ec245fd2778e..752e8277f2616 100644 --- a/net/mptcp/protocol.h +++ b/net/mptcp/protocol.h @@ -787,9 +787,7 @@ static inline bool mptcp_epollin_ready(const struct soc= k *sk) * as it can always coalesce them */ return (data_avail >=3D sk->sk_rcvlowat) || - (mem_cgroup_sockets_enabled && sk->sk_memcg && - mem_cgroup_under_socket_pressure(sk->sk_memcg)) || - READ_ONCE(tcp_memory_pressure); + tcp_under_memory_pressure(sk); } =20 int mptcp_set_rcvlowat(struct sock *sk, int val); --=20 2.50.0.727.gbf7dc18ff4-goog From nobody Wed Sep 17 18:19:40 2025 Received: from mail-pf1-f201.google.com (mail-pf1-f201.google.com [209.85.210.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6088B23C514 for ; Mon, 21 Jul 2025 20:36:32 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1753130193; cv=none; b=DBt0x/C5zlkNNTM9HYOsu9d9Vo2w1s+ClUZpmWzoMFmlB1jBJKAumnVAmuhI04fOwn99geq0EIOE0G/Qqzv8qY4XtVg4swXcm4NGQKGiJsnsH4inDDYzK2bZmioauHMD4uZfqSfeC1gqsNdVeZDx9LxnjooVr2GSMTzkG1YeD68= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1753130193; c=relaxed/simple; bh=j5N4aCPTqWCMs7i3WRNZ2KVBnmcFjVWd2rdC7aOLvuw=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=TP5F+4BxLK5rQplJwPoAcL+BCHFjwfnspc6FJu8UJgpOFxIP4uJYyhFuFs2ORj/p25wvU6Dk7WNcJMXY/Al31S7A3Da4ew1Bc/bX7rAw4YcdHHcmMpqr+/cjY0PgFd2aTw+/8LNhHnobVEOHnOSmEVpaA67o8HUFg/3FCQbzAP8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--kuniyu.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=hoGSMrK7; arc=none smtp.client-ip=209.85.210.201 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--kuniyu.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="hoGSMrK7" Received: by mail-pf1-f201.google.com with SMTP id d2e1a72fcca58-748f3d4c7e7so3949796b3a.3 for ; Mon, 21 Jul 2025 13:36:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1753130192; x=1753734992; darn=lists.linux.dev; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=Wpwn6QQouhXynO23Cu972bLk01/99pIHsLHoJU/mo78=; b=hoGSMrK7dTFMP2MI1nMiFlMYpBVFTX47lni/RtblAirHL5qGlyWuEdWFnCRkbXQqPA hEVr7E7eROGrHRyXtNAYfo4v/6Pzl7wQG9LSQlZIWbvCILbyfnlp/ZK8ZFZWz0/7m36z MuxpZWV4JKalJVFvusFdneT6BxZwwWOiRcoJyC5o+fwhp4KEmjGU6Ji1n1gVSMEwQ2Rx ln44VWhKjBP79Hx/nCetYJGOg12XZG9s/uo3fnxI4zqpMemBDiiOJV8/YpKVx3Ex7/OJ dMArnPVWa/Om/kJo7tUyqV3mhdNKYhrFVi/MZ9JPERKi8JEnRd0DBd3Uypv66wyaSd/E aTlg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1753130192; x=1753734992; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=Wpwn6QQouhXynO23Cu972bLk01/99pIHsLHoJU/mo78=; b=Ut23lArIE2R+ZZ4BpxdXgEQT3yvYs9VWv9RV7BYS6T/RLHgFEBHYNwFleVa4yUyhsQ LTRPCl+ZwxJwOXSCMiKdDddDlxDQSZIO4BMXo94WyIwn1oFPflEEAQa+iNorw0ZqLgJU xB+eQs3AD0XzwK0xlo27k9CBn4D/6STaFvRBIYyZe9NKvqU2spnQ/dDvhm+woHEbjAlr O2y0hH9XdeTBiBwlF/Q1oZMjzmhzH08yLAb7SXbPLUA8nWhL89Y4krfPeScuJlpXCzJ2 khPPa5saT40gAzKImScTxbBNostlsJNTW5wOmZYrKsGuCwOZtxA/+APbnpcvjDbc3LI9 iMow== X-Forwarded-Encrypted: i=1; AJvYcCXjqcbP9hO5YHiHoPOJfT4Xfr4gIbCJxn08jKHU64200tNKv71r8F4twuQdkT+OdlngECSlDQ==@lists.linux.dev X-Gm-Message-State: AOJu0YzKiq3pGe2NvGSivWnNEhGe4r5k7smapZDIFY38d+N3M8adjXcN hr1aHTvhNUox0/09uRf595bHBpBZp2lcm6bcGwAcI8z0b9/EZZQhlUzE6dK/Z6DCYrclCi1UbiM JIXnKPQ== X-Google-Smtp-Source: AGHT+IGOSqwwC3mh3GOf+Pwdhi6WvfqKoRK+i/P5LIuBAbC+/yAfslPdXIv1OdhrmQiErVu217W2zT+wZxQ= X-Received: from pfbml3.prod.google.com ([2002:a05:6a00:3d83:b0:748:e00c:4e66]) (user=kuniyu job=prod-delivery.src-stubby-dispatcher) by 2002:a05:6a20:7d9f:b0:232:7c7b:1c7b with SMTP id adf61e73a8af0-23810d50911mr35590008637.14.1753130191720; Mon, 21 Jul 2025 13:36:31 -0700 (PDT) Date: Mon, 21 Jul 2025 20:35:22 +0000 In-Reply-To: <20250721203624.3807041-1-kuniyu@google.com> Precedence: bulk X-Mailing-List: mptcp@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250721203624.3807041-1-kuniyu@google.com> X-Mailer: git-send-email 2.50.0.727.gbf7dc18ff4-goog Message-ID: <20250721203624.3807041-4-kuniyu@google.com> Subject: [PATCH v1 net-next 03/13] tcp: Simplify error path in inet_csk_accept(). From: Kuniyuki Iwashima To: "David S. Miller" , Eric Dumazet , Jakub Kicinski , Neal Cardwell , Paolo Abeni , Willem de Bruijn , Matthieu Baerts , Mat Martineau , Johannes Weiner , Michal Hocko , Roman Gushchin , Shakeel Butt , Andrew Morton Cc: Simon Horman , Geliang Tang , Muchun Song , Kuniyuki Iwashima , Kuniyuki Iwashima , netdev@vger.kernel.org, mptcp@lists.linux.dev, cgroups@vger.kernel.org, linux-mm@kvack.org Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" When an error occurs in inet_csk_accept(), what we should do is only call release_sock() and set the errno to arg->err. But the path jumps to another label, which introduces unnecessary initialisation and tests for newsk. Let's simplify the error path and remove the redundant NULL checks for newsk. Signed-off-by: Kuniyuki Iwashima Reviewed-by: Eric Dumazet --- net/ipv4/inet_connection_sock.c | 15 +++++++-------- 1 file changed, 7 insertions(+), 8 deletions(-) diff --git a/net/ipv4/inet_connection_sock.c b/net/ipv4/inet_connection_soc= k.c index 1e2df51427fed..724bd9ed6cd48 100644 --- a/net/ipv4/inet_connection_sock.c +++ b/net/ipv4/inet_connection_sock.c @@ -706,9 +706,9 @@ struct sock *inet_csk_accept(struct sock *sk, struct pr= oto_accept_arg *arg) spin_unlock_bh(&queue->fastopenq.lock); } =20 -out: release_sock(sk); - if (newsk && mem_cgroup_sockets_enabled) { + + if (mem_cgroup_sockets_enabled) { gfp_t gfp =3D GFP_KERNEL | __GFP_NOFAIL; int amt =3D 0; =20 @@ -732,18 +732,17 @@ struct sock *inet_csk_accept(struct sock *sk, struct = proto_accept_arg *arg) =20 release_sock(newsk); } + if (req) reqsk_put(req); =20 - if (newsk) - inet_init_csk_locks(newsk); - + inet_init_csk_locks(newsk); return newsk; + out_err: - newsk =3D NULL; - req =3D NULL; + release_sock(sk); arg->err =3D error; - goto out; + return NULL; } EXPORT_SYMBOL(inet_csk_accept); =20 --=20 2.50.0.727.gbf7dc18ff4-goog From nobody Wed Sep 17 18:19:40 2025 Received: from mail-pf1-f202.google.com (mail-pf1-f202.google.com [209.85.210.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 379AB23F294 for ; Mon, 21 Jul 2025 20:36:34 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1753130195; cv=none; b=lGqsVmteA+gACrrMB8T2oTn/G51Qdf42ZtbiIODqEk3prb0Lk2VgzVBvvUtTQ0mocUzb1DI3hN8k6otMpJIHFpFeBtGOelQcC6nzJmWawwtpsh6kZot9h2rMb4p1T061Ag47lT49UAroUCxjn0IoPQFQasW9cvNFqUPfckIG/l8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1753130195; c=relaxed/simple; bh=7GYx0gT7yKuvyKvrpKME7Js52yeuvXmMr+g6Ow0vwcM=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=HkeurCp5RtJ63vrFsu8VMk6KxMH6cklRB0vjKM7JHAruz5uO9377uo/4FFNEIMfjw5zXzycrRCu7Y3nI9u1fHBetYrBTt5Ch/QlwDwNaHKcBYHbWqn2SUqEDsPuKnkDJUMf3fi5lu6Zhwzek1cxPMkt4kWxRx5YGbMawis1+DHs= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--kuniyu.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=SD4AqN2t; arc=none smtp.client-ip=209.85.210.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--kuniyu.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="SD4AqN2t" Received: by mail-pf1-f202.google.com with SMTP id d2e1a72fcca58-756a4884dfcso4489838b3a.3 for ; Mon, 21 Jul 2025 13:36:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1753130193; x=1753734993; darn=lists.linux.dev; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=hYUyNgMVCCaofoOHBPPUoPupTfXzfLB6/aiuX2w18Mw=; b=SD4AqN2tC3MSVhV2xpW41HIU3w8GBzs6dhzTpASycksIkCt1+4UVQYT63muT8qF6BW atOz1T5XslGGx0w+NUdkJfbWEF+MJIdgKeeL954mTHkkNUaHASGmgckAcVXxlMJ89VMd cK2FTkmP6hAbQxW6hHEfDV0m2XYQzWXur1SuOeuCFlFvpm7e4Lf9LxXwylCgnq+buyGJ vFEQ84nwQUnJHvTST1lQbeeNGkixzHtQrkxQhAgTggeeG8Q1p+r9unp3d/oAjJdDzrN2 /3HWfd12xg4R89l/ZzOn2qx0cRD0EZwJtXFso3VCh5F+o0Pywm39KgWGoYvBOtEoRWGX kgTg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1753130193; x=1753734993; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=hYUyNgMVCCaofoOHBPPUoPupTfXzfLB6/aiuX2w18Mw=; b=e6djSRuWCDx7yDiENDNlvwiU4oUczyIN9t65whnhFBUH+IrxkSoaWjBiDGFRnMwKfX SF3KBTlqn25CxCIVS314np95NLjXfec6twEwahcPuVjPemvmpnoYE6+1LTeRaZu6RCXS H967PB7laS1zzfIm1Hqtp5NwREmcpf7FmCwcKzCwuKn+CSWMNQYsBZmZs/oB0zI436kC SGppq1lMIin12nDSEKxHY9fmucawOSl9rnjv7xEAHJzfIcGSQFQ1CX7T7rZTKB26vuSK EheXFnlNNZR4Zv1t8zVS9mwseMJSlmCHI2ypxrjSGRGgOd9/RFHwTd/xAvNuRTaeR1Py IJJw== X-Forwarded-Encrypted: i=1; AJvYcCV47nkH6yFOj3WRphcvRzdJIfkami4VX7/GojvphrGbYNjqIOKbimUtWsn3CYQ7l297ghWjWg==@lists.linux.dev X-Gm-Message-State: AOJu0Yy0pbOmPrDsu/gZXECgpo9LQczbJP+21wIBDn++fsF+dkccgrH9 AkMh0OrwiwN0fQwnHgPYREksEF2YxZsz52KuAH9UF/sjs7DPnrwdlriLWe8MgOsZx+rY57hizwK 7da+1OA== X-Google-Smtp-Source: AGHT+IGmJf5+70mstVkzYNSRFS4u46asIcv4vmF3QPlJsewm8h4FN3p0mxWoexnQlkXY7f0461xV86Ew6Co= X-Received: from pgll188.prod.google.com ([2002:a63:25c5:0:b0:b2c:35e1:f278]) (user=kuniyu job=prod-delivery.src-stubby-dispatcher) by 2002:a05:6a21:62c1:b0:216:1476:f5c with SMTP id adf61e73a8af0-23812b50d6cmr36295824637.25.1753130193273; Mon, 21 Jul 2025 13:36:33 -0700 (PDT) Date: Mon, 21 Jul 2025 20:35:23 +0000 In-Reply-To: <20250721203624.3807041-1-kuniyu@google.com> Precedence: bulk X-Mailing-List: mptcp@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250721203624.3807041-1-kuniyu@google.com> X-Mailer: git-send-email 2.50.0.727.gbf7dc18ff4-goog Message-ID: <20250721203624.3807041-5-kuniyu@google.com> Subject: [PATCH v1 net-next 04/13] net: Call trace_sock_exceed_buf_limit() for memcg failure with SK_MEM_RECV. From: Kuniyuki Iwashima To: "David S. Miller" , Eric Dumazet , Jakub Kicinski , Neal Cardwell , Paolo Abeni , Willem de Bruijn , Matthieu Baerts , Mat Martineau , Johannes Weiner , Michal Hocko , Roman Gushchin , Shakeel Butt , Andrew Morton Cc: Simon Horman , Geliang Tang , Muchun Song , Kuniyuki Iwashima , Kuniyuki Iwashima , netdev@vger.kernel.org, mptcp@lists.linux.dev, cgroups@vger.kernel.org, linux-mm@kvack.org Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Initially, trace_sock_exceed_buf_limit() was invoked when __sk_mem_raise_allocated() failed due to the memcg limit or the global limit. However, commit d6f19938eb031 ("net: expose sk wmem in sock_exceed_buf_limit tracepoint") somehow suppressed the event only when memcg failed to charge for SK_MEM_RECV, although the memcg failure for SK_MEM_SEND still triggers the event. Let's restore the event for SK_MEM_RECV. Signed-off-by: Kuniyuki Iwashima Reviewed-by: Eric Dumazet --- net/core/sock.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/net/core/sock.c b/net/core/sock.c index 7c26ec8dce630..380bc1aa69829 100644 --- a/net/core/sock.c +++ b/net/core/sock.c @@ -3354,8 +3354,7 @@ int __sk_mem_raise_allocated(struct sock *sk, int siz= e, int amt, int kind) } } =20 - if (kind =3D=3D SK_MEM_SEND || (kind =3D=3D SK_MEM_RECV && charged)) - trace_sock_exceed_buf_limit(sk, prot, allocated, kind); + trace_sock_exceed_buf_limit(sk, prot, allocated, kind); =20 sk_memory_allocated_sub(sk, amt); =20 --=20 2.50.0.727.gbf7dc18ff4-goog From nobody Wed Sep 17 18:19:40 2025 Received: from mail-pf1-f201.google.com (mail-pf1-f201.google.com [209.85.210.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 362C423E33A for ; Mon, 21 Jul 2025 20:36:35 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1753130196; cv=none; b=GE/ZK+9eQuB7pEWIe+R/4RXsSGxkS+c+J+ZAgurCs8Ty/4eyGFgBi5VyZIHjEASh5NOyJUIsMfmon9w7l9g4b0Doc1yekrmcbZyPSCfZu7X68GN2Jm8fF4RRXnrAs0RgNJulBAkr3EU2TmA52R2KGb2a5PjrMCzAf1Tph8LrxmQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1753130196; c=relaxed/simple; bh=x6W9fSLS0sdD6u16GfcO5OEeoEdgWbD5AwJHjH26y4g=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=FIqq/EN4TTl/yZ8pKceDRZ+d6yMNJgcsdxH8tyPmGp/9I5d5R7WWjRK8s4kFtFfpfqESKEnSDiP12hes5ESB5P1x8cCwM/TDh5BnPZf7hs5E9FWrOWN4bCQPx4CjqdMKXTHZnfd/dMq9M5aXNHMDaLOU/GM50q213XpX/dK5HyA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--kuniyu.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=GpRc0q7l; arc=none smtp.client-ip=209.85.210.201 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--kuniyu.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="GpRc0q7l" Received: by mail-pf1-f201.google.com with SMTP id d2e1a72fcca58-748f3613e6aso2293667b3a.0 for ; Mon, 21 Jul 2025 13:36:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1753130195; x=1753734995; darn=lists.linux.dev; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=I175s6NxGih3nP7OcDsLqMOyvojLdjw63UOF+xheMJ4=; b=GpRc0q7lN0gbCh2tDhOl/hxSR2QnpYzY7fbcAfmM+nkrXD6fi3VloHAbXugv9AMXQg j2vw5p2o8HVwtPEyOmTkwL7aQjJGBwQNXpP5QcLLav/Y2rpGKBpKnuoKIdyr4T4RFzt3 G5qvVn+k2u1wM5CIU6H4YhN+z3PIPXjILgBcQ61c94UcwF5W1d5KjI8SL/P41qjNK3jd 2KyGa/OBmrCozd57geF5calyzbxHBjcwdv/hqmaEFqT/sAmYD/+DVjdGyGwJCBjNF8fh s2/wS3X3wqBYIVpcXf/7BxgaowLcL2YD577BN6oewrPsxf1skSR+8q8dF+40nKyVB1Qs C06A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1753130195; x=1753734995; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=I175s6NxGih3nP7OcDsLqMOyvojLdjw63UOF+xheMJ4=; b=TfVLIOjJNhE7s9oOUnPTfO3xf2LjDz9vIESvZjK+LCbtGELg+/dWrKM13IuXqcOXGN Bqdv6ASHAHyvDyyvmpIzr7tqi5/S8WF2gS3JFR6By0WvH+KN/HsANaue0ExFdqoJ5/ps AJPg6MH9+oUTYPJ2+tTfGEIjAXCBO3Ik97JBrumUV3ns7I+7JeChKLvR46s35PZ5sBf8 4D/99U2nbSOtla4s4SCG2R8u06O6uFjgu/aCnq6qKLBjmH6dngyPjdzJJ4WjaLOi0bcX q51qTOhkcjZjHmBez805iSDCMbUWElzdKUjFvQ8Jim3j/EMBVq0mG+MReU87r2zyGHAm so1g== X-Forwarded-Encrypted: i=1; AJvYcCXXT3FKF2niHu9blK6y/hr0d18gzVBYXGgh8A1bZshHjJnSDsS+vDjwZYnY0PaC8/31ifM2KA==@lists.linux.dev X-Gm-Message-State: AOJu0YyTqMlopB6tUCRkWWiRaw0CXT3Mvv8wItVtcIbR1eGBKvzC2ldk d5SQdySLcnNHr/b5eQ/xYs8uEt9iS5lIs7Y690IYAg45y9/vxAeLWsz0Hm765VaINp3zx3is+r1 fF9Gk+w== X-Google-Smtp-Source: AGHT+IFv6U7uHL2iBa9yupFYshgEXju3eOkU6QMVmKj+fMe7znAZlxzbdQTya4Ql3pAJYEeoSFGsqqqmF3U= X-Received: from pfay32.prod.google.com ([2002:a05:6a00:1820:b0:74b:54a2:ff33]) (user=kuniyu job=prod-delivery.src-stubby-dispatcher) by 2002:a05:6a00:4f86:b0:730:75b1:7219 with SMTP id d2e1a72fcca58-7572466bee2mr28714696b3a.12.1753130194603; Mon, 21 Jul 2025 13:36:34 -0700 (PDT) Date: Mon, 21 Jul 2025 20:35:24 +0000 In-Reply-To: <20250721203624.3807041-1-kuniyu@google.com> Precedence: bulk X-Mailing-List: mptcp@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250721203624.3807041-1-kuniyu@google.com> X-Mailer: git-send-email 2.50.0.727.gbf7dc18ff4-goog Message-ID: <20250721203624.3807041-6-kuniyu@google.com> Subject: [PATCH v1 net-next 05/13] net: Clean up __sk_mem_raise_allocated(). From: Kuniyuki Iwashima To: "David S. Miller" , Eric Dumazet , Jakub Kicinski , Neal Cardwell , Paolo Abeni , Willem de Bruijn , Matthieu Baerts , Mat Martineau , Johannes Weiner , Michal Hocko , Roman Gushchin , Shakeel Butt , Andrew Morton Cc: Simon Horman , Geliang Tang , Muchun Song , Kuniyuki Iwashima , Kuniyuki Iwashima , netdev@vger.kernel.org, mptcp@lists.linux.dev, cgroups@vger.kernel.org, linux-mm@kvack.org Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" In __sk_mem_raise_allocated(), charged is initialised as true due to the weird condition removed in the previous patch. It makes the variable unreliable by itself, so we have to check another variable, memcg, in advance. Also, we will factorise the common check below for memcg later. if (mem_cgroup_sockets_enabled && sk->sk_memcg) As a prep, let's initialise charged as false and memcg as NULL. Signed-off-by: Kuniyuki Iwashima Reviewed-by: Eric Dumazet --- net/core/sock.c | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-) diff --git a/net/core/sock.c b/net/core/sock.c index 380bc1aa69829..000940ecf360e 100644 --- a/net/core/sock.c +++ b/net/core/sock.c @@ -3263,15 +3263,16 @@ EXPORT_SYMBOL(sk_wait_data); */ int __sk_mem_raise_allocated(struct sock *sk, int size, int amt, int kind) { - struct mem_cgroup *memcg =3D mem_cgroup_sockets_enabled ? sk->sk_memcg : = NULL; struct proto *prot =3D sk->sk_prot; - bool charged =3D true; + struct mem_cgroup *memcg =3D NULL; + bool charged =3D false; long allocated; =20 sk_memory_allocated_add(sk, amt); allocated =3D sk_memory_allocated(sk); =20 - if (memcg) { + if (mem_cgroup_sockets_enabled && sk->sk_memcg) { + memcg =3D sk->sk_memcg; charged =3D mem_cgroup_charge_skmem(memcg, amt, gfp_memcg_charge()); if (!charged) goto suppress_allocation; @@ -3358,7 +3359,7 @@ int __sk_mem_raise_allocated(struct sock *sk, int siz= e, int amt, int kind) =20 sk_memory_allocated_sub(sk, amt); =20 - if (memcg && charged) + if (charged) mem_cgroup_uncharge_skmem(memcg, amt); =20 return 0; --=20 2.50.0.727.gbf7dc18ff4-goog From nobody Wed Sep 17 18:19:40 2025 Received: from mail-pf1-f201.google.com (mail-pf1-f201.google.com [209.85.210.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 12A32241672 for ; Mon, 21 Jul 2025 20:36:36 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1753130198; cv=none; b=C09IoXnEY7h7qpCQU6amh45D6MwpKh8Zy0je+HyzvyLehKMroq3QezNQjZLqHaeu7TQcdSIOmHAMw6XjcVdbD6t+hh86ZEpFKHzCDbZE7ZLZzF4wQmJ/a5tGljwG/J0ih3t/1ZeMMm4ptkS+E3eBsVF2AdtCGubOWhDg5z+BZU0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1753130198; c=relaxed/simple; bh=wN2CF4G/nCvFEdVvHYYf1Bhf5FKqcQYYZZZXgtBjYY4=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=sqgLie0YEQTiCSbHdNFxmdQ2FBS3wQ9l7r9TP8OOKUPuk5OT2q9Qg751xpJsVUH0fh6z0VpfUMmxlrHV2oXqjlYjJiJRdPP1FvSQnPx2Y685NTTboJL+POWy+IdN3TloIIsfmhb/dSm9DnpAYGmC3EbRzBIPBfvzyatG8v4i6hY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--kuniyu.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=0V+801Yx; arc=none smtp.client-ip=209.85.210.201 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--kuniyu.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="0V+801Yx" Received: by mail-pf1-f201.google.com with SMTP id d2e1a72fcca58-7492da755a1so3550873b3a.1 for ; Mon, 21 Jul 2025 13:36:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1753130196; x=1753734996; darn=lists.linux.dev; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=mrGASm3CxvureKJlvs5onpPMiZoMQw1GHMHnORXuhuQ=; b=0V+801YxII0p7TrIIEyRn1Dfm8JyYxdFDz7wiNZx90APFdhHjdD6i0m3VnRuMnJqMQ Bei2f96Fm3eBCBQ7vDJN/Nr0eFji/D039XJtGQIjHAYrU82rT0mdUll1GKSHmUuzqmGD wg6Ym0Cc2kggKOUjKIkSsEdrJztPir/Cpo/MRHhpTCyZrltJ7bp5Uyhl8nDxXS+ZkpQJ F7C0xSo43G4OcYmGdARpXKqcdjzXgsImh300nIAkevMPPhVXHoU/Ib1DL++J3iTpQGha F0vxFIdJU7cUIwlrce4v8y/oMrRwAH+DBxIimCpzk38MMCwq3Yy7squj3td98LxU2Y/z oDyg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1753130196; x=1753734996; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=mrGASm3CxvureKJlvs5onpPMiZoMQw1GHMHnORXuhuQ=; b=W+ETqoRy1MkWRd3/k9cYilkpPNBtlXH2Qn3dAz3FjaWZFh98MtP/sJpP3uzzD4UuAo tfwIPJDHU7PviV133Wu/3CzgM6v0CShX2zw6gMVR0331zPNZKDGTg466U6QaGSQnDTP9 8j8Ap/WHC+PNZnjvUG71cU+3zWEs6+ZnLjQhALmpqLFD10ieNMB3398hfZshwY4DsTSo xQuQIxlqSvs2m4QGP+ZH4s+7vRMfzw495T7ucVsoqlB+/H/76VMCu9CELSJWc/iLpx64 KkPl3YXEcPXkr+/kBdeg04rWCMrUYor1y/8eQXjfil46uo/XZbSzfvkFbIajv/697KFV qoEg== X-Forwarded-Encrypted: i=1; AJvYcCVtr2KlLRInSm9sCWcgfkpoAekFYMa2Wq7Jj6fkqyzfj144TNall7ZVTIAgy21HZitjXDYSKQ==@lists.linux.dev X-Gm-Message-State: AOJu0YwxPteT7iBrgHTnGfvclFBWrQPvk2pRVm8ZSBZuOFGil9NiOYRz Ke7LgnBBKUy6MvC2g3fr9dua0O27SdYvH6G3a5SEY43CuD0QTzfwMXcpnibyLTbss/vNLQWwb0p xh0DUfg== X-Google-Smtp-Source: AGHT+IFThfJRCKKDOR0Nefiu/tzdT/WdSWlskFFKFs1Y1mRRZJXgb5GsFfKf6cD/RAM/mAjuTqVBtl3aIvc= X-Received: from pfbcw5.prod.google.com ([2002:a05:6a00:4505:b0:748:f4a1:ae2e]) (user=kuniyu job=prod-delivery.src-stubby-dispatcher) by 2002:a05:6a00:2189:b0:748:68dd:eb8c with SMTP id d2e1a72fcca58-7572487561emr30325627b3a.23.1753130196319; Mon, 21 Jul 2025 13:36:36 -0700 (PDT) Date: Mon, 21 Jul 2025 20:35:25 +0000 In-Reply-To: <20250721203624.3807041-1-kuniyu@google.com> Precedence: bulk X-Mailing-List: mptcp@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250721203624.3807041-1-kuniyu@google.com> X-Mailer: git-send-email 2.50.0.727.gbf7dc18ff4-goog Message-ID: <20250721203624.3807041-7-kuniyu@google.com> Subject: [PATCH v1 net-next 06/13] net-memcg: Introduce mem_cgroup_from_sk(). From: Kuniyuki Iwashima To: "David S. Miller" , Eric Dumazet , Jakub Kicinski , Neal Cardwell , Paolo Abeni , Willem de Bruijn , Matthieu Baerts , Mat Martineau , Johannes Weiner , Michal Hocko , Roman Gushchin , Shakeel Butt , Andrew Morton Cc: Simon Horman , Geliang Tang , Muchun Song , Kuniyuki Iwashima , Kuniyuki Iwashima , netdev@vger.kernel.org, mptcp@lists.linux.dev, cgroups@vger.kernel.org, linux-mm@kvack.org Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" We will store a flag in the lowest bit of sk->sk_memcg. Then, directly dereferencing sk->sk_memcg will be illegal, and we do not want to allow touching the raw sk->sk_memcg in many places. Let's introduce mem_cgroup_from_sk(). Other places accessing the raw sk->sk_memcg will be converted later. Note that we cannot define the helper as an inline function in memcontrol.h as we cannot access any fields of struct sock there due to circular dependency, so it is placed in sock.h. Signed-off-by: Kuniyuki Iwashima Reviewed-by: Eric Dumazet --- include/net/sock.h | 12 ++++++++++++ mm/memcontrol.c | 14 +++++++++----- net/ipv4/inet_connection_sock.c | 2 +- 3 files changed, 22 insertions(+), 6 deletions(-) diff --git a/include/net/sock.h b/include/net/sock.h index c8a4b283df6fc..811f95ea8d00c 100644 --- a/include/net/sock.h +++ b/include/net/sock.h @@ -2594,6 +2594,18 @@ static inline gfp_t gfp_memcg_charge(void) return in_softirq() ? GFP_ATOMIC : GFP_KERNEL; } =20 +#ifdef CONFIG_MEMCG +static inline struct mem_cgroup *mem_cgroup_from_sk(const struct sock *sk) +{ + return sk->sk_memcg; +} +#else +static inline struct mem_cgroup *mem_cgroup_from_sk(const struct sock *sk) +{ + return NULL; +} +#endif + static inline long sock_rcvtimeo(const struct sock *sk, bool noblock) { return noblock ? 0 : READ_ONCE(sk->sk_rcvtimeo); diff --git a/mm/memcontrol.c b/mm/memcontrol.c index 54eb25d8d555c..89b33e635cf89 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -5086,18 +5086,22 @@ void mem_cgroup_sk_alloc(struct sock *sk) =20 void mem_cgroup_sk_free(struct sock *sk) { - if (sk->sk_memcg) - css_put(&sk->sk_memcg->css); + struct mem_cgroup *memcg =3D mem_cgroup_from_sk(sk); + + if (memcg) + css_put(&memcg->css); } =20 void mem_cgroup_sk_inherit(const struct sock *sk, struct sock *newsk) { - if (sk->sk_memcg =3D=3D newsk->sk_memcg) + struct mem_cgroup *memcg =3D mem_cgroup_from_sk(sk); + + if (memcg =3D=3D mem_cgroup_from_sk(newsk)) return; =20 mem_cgroup_sk_free(newsk); - css_get(&sk->sk_memcg->css); - newsk->sk_memcg =3D sk->sk_memcg; + css_get(&memcg->css); + newsk->sk_memcg =3D memcg; } =20 /** diff --git a/net/ipv4/inet_connection_sock.c b/net/ipv4/inet_connection_soc= k.c index 724bd9ed6cd48..93569bbe00f44 100644 --- a/net/ipv4/inet_connection_sock.c +++ b/net/ipv4/inet_connection_sock.c @@ -718,7 +718,7 @@ struct sock *inet_csk_accept(struct sock *sk, struct pr= oto_accept_arg *arg) lock_sock(newsk); =20 mem_cgroup_sk_alloc(newsk); - if (newsk->sk_memcg) { + if (mem_cgroup_from_sk(newsk)) { /* The socket has not been accepted yet, no need * to look at newsk->sk_wmem_queued. */ --=20 2.50.0.727.gbf7dc18ff4-goog From nobody Wed Sep 17 18:19:40 2025 Received: from mail-pj1-f73.google.com (mail-pj1-f73.google.com [209.85.216.73]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9866D241695 for ; Mon, 21 Jul 2025 20:36:38 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.73 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1753130200; cv=none; b=mgTorPRaAN6la5MKaIa4PvI8K+G/SdB2Ro3ijWtx/dhVnC+38B4AOuT8h/7vFv/6bKRGoSXQEQuM2qRy2JGUDpmxPWVU99b/YfeLXaeYM3z5JlPT4j+d/l1MY+kvSnacdFMubF/Mkf//e6sM26YJ3gHWhcza88WTzUH03HC2qSo= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1753130200; c=relaxed/simple; bh=Np33N9r0abllKVI2quaOtsW8GKG8UR/+pGRXFZjdmVg=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=mFX+xjKKUitlsCgDE/JyVxrM+43WfL+k0ArJW58rGEXhEVnYzhKkXQ6k5FQ31+TQoQwh8mzw5XIBegI+xIjjO/TLXdWx9pnT0zvaIZvlT8lAU/0lnSREMV46lmaDfU18A7kXPhaPJm7I2lPob4GAq+/BTpy7/faT78l3x25chFw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--kuniyu.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=m9zXSvtC; arc=none smtp.client-ip=209.85.216.73 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--kuniyu.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="m9zXSvtC" Received: by mail-pj1-f73.google.com with SMTP id 98e67ed59e1d1-311ef4fb5eeso4608966a91.1 for ; Mon, 21 Jul 2025 13:36:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1753130198; x=1753734998; darn=lists.linux.dev; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=vzkCcE7y2iiEKWHZNkTKmLOP5up72GHF++8nWGJ0fgE=; b=m9zXSvtCuCoWmlCU6UMFlXa4bq1emtZhcIuyR2eBfLEiRrlvmvKWW6r2N+HvAQju7a 7etM9RrFobifKkq68gEFW9Ji/kCk1kWqGkn4S9s5brMJWv6MIHYH7+lo2+D5lEw/0ndU kVVjyO04RD7TgDIwY+SE/9PZa4wBqaZjXDKs0z/eZxLofK86FE+T5HubzbHZbWTL0XTp jRWXMGgGZoIwnQXzUmRhdI90gN8A6JJnCkwxmW1EIuXfi1ywGwau6EXK14hOe+VSe+Hi 24b0LQYvkBRqu4rRKbxJkgJncviiPF4c0+ckKsr3FHTihLKXBSJcURbFHnAEhtIc9SrM JNxw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1753130198; x=1753734998; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=vzkCcE7y2iiEKWHZNkTKmLOP5up72GHF++8nWGJ0fgE=; b=FM/qRgzN6IAwDFvVtErGd1Lk/hoMBht1ZiueAIHHPC0nLIu5DpD3iocSVWFkMrGrPE ZOh6DbrH+5bCS3ehRjHlLK0EusCbV8gVsGQd6+FGcBB2/PsQHTGc81xRb6/fBpNFp65D dXaqReSY4FLcLCPv+YTQ1SFZMRpTU0KuzYT/awbQEfKq7SgJSeFDnh9FtkCgyXqgPNfq zpSD2+iEYuSj660NhEKl5xaEON8Ko9FoSFQqsSxY/j1965a/vnBfYaqkXRi2KM3zUskj 6ySqe8OiQ+r/6n58xO++AQEO20ZDDq5XhHBkHmOwDrYPEDUygW8+zTwaQrjd3YxYZVA9 Y7sQ== X-Forwarded-Encrypted: i=1; AJvYcCVuwzcVqwzGCCV7BCmjIqbr8hRJynxP2B0bHk9/BFVfdRhvXOuqdz+WrknbFspSfnXlFnkLCw==@lists.linux.dev X-Gm-Message-State: AOJu0YzTLauFOzk1U06i1U3JoCFrcdaGeuiuGu6b+E8TvnECmVrMlI73 CPgaQI7oER/dZJzi+PdPqaxFMhlY3oOAlop4GWSps7ypQ7tU5VcIj/0jBIgatiknOb1Jq89N+vA 1N98Nhg== X-Google-Smtp-Source: AGHT+IE6OSr24etnKRAVeW0ucA3S4ktB5FCi5MdQH2kT6HggNuPbZhFVkCzC2ipGvzXe9c/IyTBVfm1pHvo= X-Received: from pjbhk17.prod.google.com ([2002:a17:90b:2251:b0:31e:3c57:ffc8]) (user=kuniyu job=prod-delivery.src-stubby-dispatcher) by 2002:a17:90b:3e8c:b0:310:c8ec:4192 with SMTP id 98e67ed59e1d1-31e3e1cf38fmr1125302a91.10.1753130197825; Mon, 21 Jul 2025 13:36:37 -0700 (PDT) Date: Mon, 21 Jul 2025 20:35:26 +0000 In-Reply-To: <20250721203624.3807041-1-kuniyu@google.com> Precedence: bulk X-Mailing-List: mptcp@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250721203624.3807041-1-kuniyu@google.com> X-Mailer: git-send-email 2.50.0.727.gbf7dc18ff4-goog Message-ID: <20250721203624.3807041-8-kuniyu@google.com> Subject: [PATCH v1 net-next 07/13] net-memcg: Introduce mem_cgroup_sk_enabled(). From: Kuniyuki Iwashima To: "David S. Miller" , Eric Dumazet , Jakub Kicinski , Neal Cardwell , Paolo Abeni , Willem de Bruijn , Matthieu Baerts , Mat Martineau , Johannes Weiner , Michal Hocko , Roman Gushchin , Shakeel Butt , Andrew Morton Cc: Simon Horman , Geliang Tang , Muchun Song , Kuniyuki Iwashima , Kuniyuki Iwashima , netdev@vger.kernel.org, mptcp@lists.linux.dev, cgroups@vger.kernel.org, linux-mm@kvack.org Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" The socket memcg feature is enabled by a static key and only works for non-root cgroup. We check both conditions in many places. Let's factorise it as a helper function. Signed-off-by: Kuniyuki Iwashima Reviewed-by: Eric Dumazet --- include/net/proto_memory.h | 2 +- include/net/sock.h | 10 ++++++++++ include/net/tcp.h | 2 +- net/core/sock.c | 6 +++--- net/ipv4/tcp_output.c | 2 +- net/mptcp/subflow.c | 2 +- 6 files changed, 17 insertions(+), 7 deletions(-) diff --git a/include/net/proto_memory.h b/include/net/proto_memory.h index a6ab2f4f5e28a..859e63de81c49 100644 --- a/include/net/proto_memory.h +++ b/include/net/proto_memory.h @@ -31,7 +31,7 @@ static inline bool sk_under_memory_pressure(const struct = sock *sk) if (!sk->sk_prot->memory_pressure) return false; =20 - if (mem_cgroup_sockets_enabled && sk->sk_memcg && + if (mem_cgroup_sk_enabled(sk) && mem_cgroup_under_socket_pressure(sk->sk_memcg)) return true; =20 diff --git a/include/net/sock.h b/include/net/sock.h index 811f95ea8d00c..3efdf680401dd 100644 --- a/include/net/sock.h +++ b/include/net/sock.h @@ -2599,11 +2599,21 @@ static inline struct mem_cgroup *mem_cgroup_from_sk= (const struct sock *sk) { return sk->sk_memcg; } + +static inline bool mem_cgroup_sk_enabled(const struct sock *sk) +{ + return mem_cgroup_sockets_enabled && mem_cgroup_from_sk(sk); +} #else static inline struct mem_cgroup *mem_cgroup_from_sk(const struct sock *sk) { return NULL; } + +static inline bool mem_cgroup_sk_enabled(const struct sock *sk) +{ + return false; +} #endif =20 static inline long sock_rcvtimeo(const struct sock *sk, bool noblock) diff --git a/include/net/tcp.h b/include/net/tcp.h index b3815d1043400..f9a0eb242e65c 100644 --- a/include/net/tcp.h +++ b/include/net/tcp.h @@ -275,7 +275,7 @@ extern unsigned long tcp_memory_pressure; /* optimized version of sk_under_memory_pressure() for TCP sockets */ static inline bool tcp_under_memory_pressure(const struct sock *sk) { - if (mem_cgroup_sockets_enabled && sk->sk_memcg && + if (mem_cgroup_sk_enabled(sk) && mem_cgroup_under_socket_pressure(sk->sk_memcg)) return true; =20 diff --git a/net/core/sock.c b/net/core/sock.c index 000940ecf360e..ab658fe23e1e6 100644 --- a/net/core/sock.c +++ b/net/core/sock.c @@ -1032,7 +1032,7 @@ static int sock_reserve_memory(struct sock *sk, int b= ytes) bool charged; int pages; =20 - if (!mem_cgroup_sockets_enabled || !sk->sk_memcg || !sk_has_account(sk)) + if (!mem_cgroup_sk_enabled(sk) || !sk_has_account(sk)) return -EOPNOTSUPP; =20 if (!bytes) @@ -3271,7 +3271,7 @@ int __sk_mem_raise_allocated(struct sock *sk, int siz= e, int amt, int kind) sk_memory_allocated_add(sk, amt); allocated =3D sk_memory_allocated(sk); =20 - if (mem_cgroup_sockets_enabled && sk->sk_memcg) { + if (mem_cgroup_sk_enabled(sk)) { memcg =3D sk->sk_memcg; charged =3D mem_cgroup_charge_skmem(memcg, amt, gfp_memcg_charge()); if (!charged) @@ -3398,7 +3398,7 @@ void __sk_mem_reduce_allocated(struct sock *sk, int a= mount) { sk_memory_allocated_sub(sk, amount); =20 - if (mem_cgroup_sockets_enabled && sk->sk_memcg) + if (mem_cgroup_sk_enabled(sk)) mem_cgroup_uncharge_skmem(sk->sk_memcg, amount); =20 if (sk_under_global_memory_pressure(sk) && diff --git a/net/ipv4/tcp_output.c b/net/ipv4/tcp_output.c index b616776e3354c..4e0af5c824c1a 100644 --- a/net/ipv4/tcp_output.c +++ b/net/ipv4/tcp_output.c @@ -3566,7 +3566,7 @@ void sk_forced_mem_schedule(struct sock *sk, int size) sk_forward_alloc_add(sk, amt << PAGE_SHIFT); sk_memory_allocated_add(sk, amt); =20 - if (mem_cgroup_sockets_enabled && sk->sk_memcg) + if (mem_cgroup_sk_enabled(sk)) mem_cgroup_charge_skmem(sk->sk_memcg, amt, gfp_memcg_charge() | __GFP_NOFAIL); } diff --git a/net/mptcp/subflow.c b/net/mptcp/subflow.c index f21d90fb1a19d..5325642bcbbce 100644 --- a/net/mptcp/subflow.c +++ b/net/mptcp/subflow.c @@ -1722,7 +1722,7 @@ static void mptcp_attach_cgroup(struct sock *parent, = struct sock *child) } #endif /* CONFIG_SOCK_CGROUP_DATA */ =20 - if (mem_cgroup_sockets_enabled && parent->sk_memcg) + if (mem_cgroup_sk_enabled(parent)) mem_cgroup_sk_inherit(parent, child); } =20 --=20 2.50.0.727.gbf7dc18ff4-goog From nobody Wed Sep 17 18:19:40 2025 Received: from mail-pj1-f73.google.com (mail-pj1-f73.google.com [209.85.216.73]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C205B241CA2 for ; Mon, 21 Jul 2025 20:36:39 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.73 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1753130202; cv=none; b=ngTAJ7eNUvknNnAKQmQuuK8KWC52QnuosvFhGvbKV4cW3kRv/Yisr9W7Lp3p8xHpfFVxYHm3trmYp5WGhY+YdquEog1S+W3cRrpvqYdbWaTaFUREvN9QhHJs8EOZEDZCcZeTE1o87XGqUOQx3whWEHi0mPZbJyrVw3u3Uk1KRaY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1753130202; c=relaxed/simple; bh=PTX3iVv9Ku7TSrw69jK0X11zqasCKUo1GUoDKQ1jgFk=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=NtkeB0soy4Ds3QbDtdw5aXrY72F1KsjrAPbPfyWaDuDHtYX4Yc0PdOHKjUVe/tean0JLz+rdQFrLeLxY5X9TtHQKr4m9+ra5nfLlxn9u83YeOKDUJFTHm+vhcW/TczEyhXX7wmHGOxEstIsCuESeu3LVf3TXrrgLUx9ac2YbcIE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--kuniyu.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=WK/gTyhF; arc=none smtp.client-ip=209.85.216.73 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--kuniyu.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="WK/gTyhF" Received: by mail-pj1-f73.google.com with SMTP id 98e67ed59e1d1-311d670ad35so4385765a91.3 for ; Mon, 21 Jul 2025 13:36:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1753130199; x=1753734999; darn=lists.linux.dev; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=AoUvWeKsVeVx1EY4fRbQ+6aQfQTe9Vq1LkXAKDABLBI=; b=WK/gTyhF1VR3T07z1XYnDHcNxZAbVrDgKfhSydmfq+23gFUuv2jUKFe/xzZ7Q3b6Ak FBxzgpd2l5Xhac/gZKAbA/Aclar+8tDz0aKln/e3V3joSMC71c/rZX1+LhwfRoFQiagh jitgLdeVoo7NhUeASGprIbk8wUCbAQ/os4NHVGL0DKvc3L4KWfQ3XzvFlKkOOTFgDGkT ntxl/L7uVFJ5VF8mPR+wVH/79SG88ieswMhxiHYhep/+DzG4VfRoCsy8nYmAYcj0ETU1 VBmHoptMSraUVPZZfzohixJuJtIYvnt6zIp/ta+pyyrQrQUTyAJsTGbasQqg5/o30mXl 4Nzg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1753130199; x=1753734999; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=AoUvWeKsVeVx1EY4fRbQ+6aQfQTe9Vq1LkXAKDABLBI=; b=vdyz/BeiEnCvaJxWphBawSXuKPI4uzKuCvhDh4mNzIh84X5h08fThlM6ykDmIsp9Tr BF3HdepCQNosV4KhIJR1kOZcYkSZPnsIe/lMG0U7v+w1cdjlBD+dKhvq3RnQR9XDnqgI 4oT9/w7h/9VWGAJdkW0YpphXR3rqhGXCKF7V6jBgO8ewQj+KhGcaq8mgrVpPvWaiEPl4 oTqXyhTba5uKgDN8w7H0nAf86Ah5E21M7E/qu/U3b1V7l4Z5oSi1vAGTg1Dkk0wkPjau M4QqCG+Lc41XrnjvPky78R119fc6UjaNqea+wS3Lnp74uXf9VG4saAqzOLTItgTSSSKd EB/w== X-Forwarded-Encrypted: i=1; AJvYcCV0snEBfbhifO/22xXvaRuowP62heiDwju+ULYya8HHUROkfW3EINZpaRvhXylhkb577OeY8g==@lists.linux.dev X-Gm-Message-State: AOJu0YzCqWmH9+TuJQNXKno+tFJCFm7iaIbpYVtY3OiRFeiye6SAvg5L ME3ikuMxWxcK8B/an4gGlZfPZAnmjJPZJxCvS4G+hyH9vmI9NZIKjx8LKOab6GkYFjjhnvcaZwK 4uIcZRw== X-Google-Smtp-Source: AGHT+IF0GEZfGU8amzJJ0YhldecloQOnsxr/9oopxqYyUHJO4PqIqFZMmRr7D3sidRtGioBbSwED02P8YQM= X-Received: from pjzz15.prod.google.com ([2002:a17:90b:58ef:b0:311:ef56:7694]) (user=kuniyu job=prod-delivery.src-stubby-dispatcher) by 2002:a17:90b:5385:b0:313:d343:4e09 with SMTP id 98e67ed59e1d1-31c9f45e1a3mr25469971a91.3.1753130199131; Mon, 21 Jul 2025 13:36:39 -0700 (PDT) Date: Mon, 21 Jul 2025 20:35:27 +0000 In-Reply-To: <20250721203624.3807041-1-kuniyu@google.com> Precedence: bulk X-Mailing-List: mptcp@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250721203624.3807041-1-kuniyu@google.com> X-Mailer: git-send-email 2.50.0.727.gbf7dc18ff4-goog Message-ID: <20250721203624.3807041-9-kuniyu@google.com> Subject: [PATCH v1 net-next 08/13] net-memcg: Pass struct sock to mem_cgroup_sk_(un)?charge(). From: Kuniyuki Iwashima To: "David S. Miller" , Eric Dumazet , Jakub Kicinski , Neal Cardwell , Paolo Abeni , Willem de Bruijn , Matthieu Baerts , Mat Martineau , Johannes Weiner , Michal Hocko , Roman Gushchin , Shakeel Butt , Andrew Morton Cc: Simon Horman , Geliang Tang , Muchun Song , Kuniyuki Iwashima , Kuniyuki Iwashima , netdev@vger.kernel.org, mptcp@lists.linux.dev, cgroups@vger.kernel.org, linux-mm@kvack.org Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" We will store a flag in the lowest bit of sk->sk_memcg. Then, we cannot pass the raw pointer to mem_cgroup_charge_skmem() and mem_cgroup_uncharge_skmem(). Let's pass struct sock to the functions. While at it, they are renamed to match other functions starting with mem_cgroup_sk_. Signed-off-by: Kuniyuki Iwashima Reviewed-by: Eric Dumazet --- include/linux/memcontrol.h | 29 ++++++++++++++++++++++++----- mm/memcontrol.c | 18 +++++++++++------- net/core/sock.c | 24 +++++++++++------------- net/ipv4/inet_connection_sock.c | 2 +- net/ipv4/tcp_output.c | 3 +-- 5 files changed, 48 insertions(+), 28 deletions(-) diff --git a/include/linux/memcontrol.h b/include/linux/memcontrol.h index d8319ad5e8ea7..9ccbcddbe3b8e 100644 --- a/include/linux/memcontrol.h +++ b/include/linux/memcontrol.h @@ -1594,15 +1594,16 @@ static inline void mem_cgroup_flush_foreign(struct = bdi_writeback *wb) #endif /* CONFIG_CGROUP_WRITEBACK */ =20 struct sock; -bool mem_cgroup_charge_skmem(struct mem_cgroup *memcg, unsigned int nr_pag= es, - gfp_t gfp_mask); -void mem_cgroup_uncharge_skmem(struct mem_cgroup *memcg, unsigned int nr_p= ages); #ifdef CONFIG_MEMCG extern struct static_key_false memcg_sockets_enabled_key; #define mem_cgroup_sockets_enabled static_branch_unlikely(&memcg_sockets_e= nabled_key) + void mem_cgroup_sk_alloc(struct sock *sk); void mem_cgroup_sk_free(struct sock *sk); void mem_cgroup_sk_inherit(const struct sock *sk, struct sock *newsk); +bool mem_cgroup_sk_charge(const struct sock *sk, unsigned int nr_pages, + gfp_t gfp_mask); +void mem_cgroup_sk_uncharge(const struct sock *sk, unsigned int nr_pages); =20 static inline bool mem_cgroup_under_socket_pressure(struct mem_cgroup *mem= cg) { @@ -1623,13 +1624,31 @@ void set_shrinker_bit(struct mem_cgroup *memcg, int= nid, int shrinker_id); void reparent_shrinker_deferred(struct mem_cgroup *memcg); #else #define mem_cgroup_sockets_enabled 0 -static inline void mem_cgroup_sk_alloc(struct sock *sk) { }; -static inline void mem_cgroup_sk_free(struct sock *sk) { }; + +static inline void mem_cgroup_sk_alloc(struct sock *sk) +{ +} + +static inline void mem_cgroup_sk_free(struct sock *sk) +{ +} =20 static inline void mem_cgroup_sk_inherit(const struct sock *sk, struct soc= k *newsk) { } =20 +static inline bool mem_cgroup_sk_charge(const struct sock *sk, + unsigned int nr_pages, + gfp_t gfp_mask) +{ + return false; +} + +static inline void mem_cgroup_sk_uncharge(const struct sock *sk, + unsigned int nr_pages) +{ +} + static inline bool mem_cgroup_under_socket_pressure(struct mem_cgroup *mem= cg) { return false; diff --git a/mm/memcontrol.c b/mm/memcontrol.c index 89b33e635cf89..d7f4e31f4e625 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -5105,17 +5105,19 @@ void mem_cgroup_sk_inherit(const struct sock *sk, s= truct sock *newsk) } =20 /** - * mem_cgroup_charge_skmem - charge socket memory - * @memcg: memcg to charge + * mem_cgroup_sk_charge - charge socket memory + * @sk: socket in memcg to charge * @nr_pages: number of pages to charge * @gfp_mask: reclaim mode * * Charges @nr_pages to @memcg. Returns %true if the charge fit within * @memcg's configured limit, %false if it doesn't. */ -bool mem_cgroup_charge_skmem(struct mem_cgroup *memcg, unsigned int nr_pag= es, - gfp_t gfp_mask) +bool mem_cgroup_sk_charge(const struct sock *sk, unsigned int nr_pages, + gfp_t gfp_mask) { + struct mem_cgroup *memcg =3D mem_cgroup_from_sk(sk); + if (!cgroup_subsys_on_dfl(memory_cgrp_subsys)) return memcg1_charge_skmem(memcg, nr_pages, gfp_mask); =20 @@ -5128,12 +5130,14 @@ bool mem_cgroup_charge_skmem(struct mem_cgroup *mem= cg, unsigned int nr_pages, } =20 /** - * mem_cgroup_uncharge_skmem - uncharge socket memory - * @memcg: memcg to uncharge + * mem_cgroup_sk_uncharge - uncharge socket memory + * @sk: socket in memcg to uncharge * @nr_pages: number of pages to uncharge */ -void mem_cgroup_uncharge_skmem(struct mem_cgroup *memcg, unsigned int nr_p= ages) +void mem_cgroup_sk_uncharge(const struct sock *sk, unsigned int nr_pages) { + struct mem_cgroup *memcg =3D mem_cgroup_from_sk(sk); + if (!cgroup_subsys_on_dfl(memory_cgrp_subsys)) { memcg1_uncharge_skmem(memcg, nr_pages); return; diff --git a/net/core/sock.c b/net/core/sock.c index ab658fe23e1e6..5537ca2638588 100644 --- a/net/core/sock.c +++ b/net/core/sock.c @@ -1041,8 +1041,8 @@ static int sock_reserve_memory(struct sock *sk, int b= ytes) pages =3D sk_mem_pages(bytes); =20 /* pre-charge to memcg */ - charged =3D mem_cgroup_charge_skmem(sk->sk_memcg, pages, - GFP_KERNEL | __GFP_RETRY_MAYFAIL); + charged =3D mem_cgroup_sk_charge(sk, pages, + GFP_KERNEL | __GFP_RETRY_MAYFAIL); if (!charged) return -ENOMEM; =20 @@ -1054,7 +1054,7 @@ static int sock_reserve_memory(struct sock *sk, int b= ytes) */ if (allocated > sk_prot_mem_limits(sk, 1)) { sk_memory_allocated_sub(sk, pages); - mem_cgroup_uncharge_skmem(sk->sk_memcg, pages); + mem_cgroup_sk_uncharge(sk, pages); return -ENOMEM; } sk_forward_alloc_add(sk, pages << PAGE_SHIFT); @@ -3263,17 +3263,16 @@ EXPORT_SYMBOL(sk_wait_data); */ int __sk_mem_raise_allocated(struct sock *sk, int size, int amt, int kind) { + bool memcg_enabled =3D false, charged =3D false; struct proto *prot =3D sk->sk_prot; - struct mem_cgroup *memcg =3D NULL; - bool charged =3D false; long allocated; =20 sk_memory_allocated_add(sk, amt); allocated =3D sk_memory_allocated(sk); =20 if (mem_cgroup_sk_enabled(sk)) { - memcg =3D sk->sk_memcg; - charged =3D mem_cgroup_charge_skmem(memcg, amt, gfp_memcg_charge()); + memcg_enabled =3D true; + charged =3D mem_cgroup_sk_charge(sk, amt, gfp_memcg_charge()); if (!charged) goto suppress_allocation; } @@ -3347,10 +3346,9 @@ int __sk_mem_raise_allocated(struct sock *sk, int si= ze, int amt, int kind) */ if (sk->sk_wmem_queued + size >=3D sk->sk_sndbuf) { /* Force charge with __GFP_NOFAIL */ - if (memcg && !charged) { - mem_cgroup_charge_skmem(memcg, amt, - gfp_memcg_charge() | __GFP_NOFAIL); - } + if (memcg_enabled && !charged) + mem_cgroup_sk_charge(sk, amt, + gfp_memcg_charge() | __GFP_NOFAIL); return 1; } } @@ -3360,7 +3358,7 @@ int __sk_mem_raise_allocated(struct sock *sk, int siz= e, int amt, int kind) sk_memory_allocated_sub(sk, amt); =20 if (charged) - mem_cgroup_uncharge_skmem(memcg, amt); + mem_cgroup_sk_uncharge(sk, amt); =20 return 0; } @@ -3399,7 +3397,7 @@ void __sk_mem_reduce_allocated(struct sock *sk, int a= mount) sk_memory_allocated_sub(sk, amount); =20 if (mem_cgroup_sk_enabled(sk)) - mem_cgroup_uncharge_skmem(sk->sk_memcg, amount); + mem_cgroup_sk_uncharge(sk, amount); =20 if (sk_under_global_memory_pressure(sk) && (sk_memory_allocated(sk) < sk_prot_mem_limits(sk, 0))) diff --git a/net/ipv4/inet_connection_sock.c b/net/ipv4/inet_connection_soc= k.c index 93569bbe00f44..0ef1eacd539d1 100644 --- a/net/ipv4/inet_connection_sock.c +++ b/net/ipv4/inet_connection_sock.c @@ -727,7 +727,7 @@ struct sock *inet_csk_accept(struct sock *sk, struct pr= oto_accept_arg *arg) } =20 if (amt) - mem_cgroup_charge_skmem(newsk->sk_memcg, amt, gfp); + mem_cgroup_sk_charge(newsk, amt, gfp); kmem_cache_charge(newsk, gfp); =20 release_sock(newsk); diff --git a/net/ipv4/tcp_output.c b/net/ipv4/tcp_output.c index 4e0af5c824c1a..09f0802f36afa 100644 --- a/net/ipv4/tcp_output.c +++ b/net/ipv4/tcp_output.c @@ -3567,8 +3567,7 @@ void sk_forced_mem_schedule(struct sock *sk, int size) sk_memory_allocated_add(sk, amt); =20 if (mem_cgroup_sk_enabled(sk)) - mem_cgroup_charge_skmem(sk->sk_memcg, amt, - gfp_memcg_charge() | __GFP_NOFAIL); + mem_cgroup_sk_charge(sk, amt, gfp_memcg_charge() | __GFP_NOFAIL); } =20 /* Send a FIN. The caller locks the socket for us. --=20 2.50.0.727.gbf7dc18ff4-goog From nobody Wed Sep 17 18:19:40 2025 Received: from mail-pj1-f73.google.com (mail-pj1-f73.google.com [209.85.216.73]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 4146F24292E for ; Mon, 21 Jul 2025 20:36:41 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.73 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1753130202; cv=none; b=J+rRJyh4edhxIalW337kYKa+C+7jiSMeebdhOoldVEs5TkDJlFX5R0ut5giAXzxL1nMkGUCe/11cMpsuGRB0t7CXiLsbCSt4JKhyo7iWI0ujFTL280QDYdjq6s0gRUOgm9FbHcOh7L//hgHfCk8N2QzNAsmHPWLQfZ9IrV4ZfUI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1753130202; c=relaxed/simple; bh=vZGJ9wL2tKP1C/bt03aiYtXRl3GT/mAQoyMnOiZ1Zas=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=Dsi7S9SUcuFuJt1CJJQ5L+ACG3m1uS9O4Ugc5qJ7oVoryaWruNnDaxUv/B47vuGUkw5PXeb9uxruiyvin79W8Qm8JRUG52zrCj+AeoISdsit++GbVXbOvzH76hQukYjm2P/NeumEjKEWGKPf6u4uPJvkPAkV25YJ5wCK4FW1fBA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--kuniyu.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=x4XZG5o4; arc=none smtp.client-ip=209.85.216.73 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--kuniyu.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="x4XZG5o4" Received: by mail-pj1-f73.google.com with SMTP id 98e67ed59e1d1-3138f5e8ff5so4826825a91.3 for ; Mon, 21 Jul 2025 13:36:41 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1753130201; x=1753735001; darn=lists.linux.dev; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=y8n9gDKu0Z+DKg5LJ3hwVCmIJaGqgWwDHBafRHDkb0E=; b=x4XZG5o4RF1Z0bBe3xl3OMfFQLTqiTF3tkGgBsWhS7kwNQX+QUxuu1TFpGhfPvt6tO M42LTcKm8FNrft99JpCmv6PoWCU1DNnYknq3NAuchLcXlyRYy7YlhhzK4XbSCA1MuEL/ r7fMDfRQsG6p1LJBMUpBdaG78nKiEuKZ97pstuPu8+m5A0j9nqc+r+i31oPhYd70I1qB Ae3N+xYL+0BcYxBZMatCRCsCxdd//5Rxxmwxm94PXFcwvEqEv6Lw1gITEJtzY6dgoUj1 DrNz3eCH1CmVNGUbwAY7Lp91EhJ9QPC2OOdThMDrdEXHoESnbXgn3IXMn8ncv3H1Che2 gMyw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1753130201; x=1753735001; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=y8n9gDKu0Z+DKg5LJ3hwVCmIJaGqgWwDHBafRHDkb0E=; b=aVTYHHa8csKu8drugBfVtICRsM6toPRsHzbX2MupLL+8CEMqwfy9H7zcaFiEHNcBj2 aybffNClWNempiNE+Ui+TRalPo1BVm1MTLlnsZ89pq0huShkJGJikH2ibyl0T9hdsq1P zpv8ay07tfMhSOcgdfDGlL++vAXhlqIYlf/twCRC9/xUYWL3+a+42oIaOR91DgJ2ei+D jONpb5nWSDE86hhJtbfb4IyB/0s4i5V02MvqNIRuxGw5mZUVTMMdU+3j7GzAicmvWjud aidF+yp5Rzr3rmaBa15jyIJU70HNEiJbOvy7RYBNa7r4P5Q4dDIMuzR8f2NZx8AZrsaS 5ckA== X-Forwarded-Encrypted: i=1; AJvYcCVzCSoiUu6cGUTvw6L0S8MK6FKNmbZpMhEyAvmVkaXbdES3twAPvnWfqtz1H9Ih2yric+0JpA==@lists.linux.dev X-Gm-Message-State: AOJu0Yw5IDgGyMELtGc4qZeu1ziZAm8aXSXVcuwEoUTz0LQ5HGk31zVo KXm7TP5vSlLM0d2nnWuS8O8Ie2xbvt+hQM1At5AJUxv0ErwPtW04j/9ppcE4bhrVs/YoDsqr5QA Zr3PpsQ== X-Google-Smtp-Source: AGHT+IFqBDBMTU4J8DYFBRw9/E8jIm7feevzb9voFXceLD2erJ4yjibft5LsmkvUX4Te4RL5m3uzw1eIikw= X-Received: from pjb8.prod.google.com ([2002:a17:90b:2f08:b0:311:a879:981f]) (user=kuniyu job=prod-delivery.src-stubby-dispatcher) by 2002:a17:90b:544f:b0:312:26d9:d5b2 with SMTP id 98e67ed59e1d1-31c9f2b5292mr34202138a91.0.1753130200564; Mon, 21 Jul 2025 13:36:40 -0700 (PDT) Date: Mon, 21 Jul 2025 20:35:28 +0000 In-Reply-To: <20250721203624.3807041-1-kuniyu@google.com> Precedence: bulk X-Mailing-List: mptcp@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250721203624.3807041-1-kuniyu@google.com> X-Mailer: git-send-email 2.50.0.727.gbf7dc18ff4-goog Message-ID: <20250721203624.3807041-10-kuniyu@google.com> Subject: [PATCH v1 net-next 09/13] net-memcg: Pass struct sock to mem_cgroup_sk_under_memory_pressure(). From: Kuniyuki Iwashima To: "David S. Miller" , Eric Dumazet , Jakub Kicinski , Neal Cardwell , Paolo Abeni , Willem de Bruijn , Matthieu Baerts , Mat Martineau , Johannes Weiner , Michal Hocko , Roman Gushchin , Shakeel Butt , Andrew Morton Cc: Simon Horman , Geliang Tang , Muchun Song , Kuniyuki Iwashima , Kuniyuki Iwashima , netdev@vger.kernel.org, mptcp@lists.linux.dev, cgroups@vger.kernel.org, linux-mm@kvack.org Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" We will store a flag in the lowest bit of sk->sk_memcg. Then, we cannot pass the raw pointer to mem_cgroup_under_socket_pressure(). Let's pass struct sock to it and rename the function to match other functions starting with mem_cgroup_sk_. Note that the helper is moved to sock.h to use mem_cgroup_from_sk(). Signed-off-by: Kuniyuki Iwashima Reviewed-by: Eric Dumazet --- include/linux/memcontrol.h | 18 ------------------ include/net/proto_memory.h | 2 +- include/net/sock.h | 21 +++++++++++++++++++++ include/net/tcp.h | 2 +- 4 files changed, 23 insertions(+), 20 deletions(-) diff --git a/include/linux/memcontrol.h b/include/linux/memcontrol.h index 9ccbcddbe3b8e..211712ec57d1a 100644 --- a/include/linux/memcontrol.h +++ b/include/linux/memcontrol.h @@ -1605,19 +1605,6 @@ bool mem_cgroup_sk_charge(const struct sock *sk, uns= igned int nr_pages, gfp_t gfp_mask); void mem_cgroup_sk_uncharge(const struct sock *sk, unsigned int nr_pages); =20 -static inline bool mem_cgroup_under_socket_pressure(struct mem_cgroup *mem= cg) -{ -#ifdef CONFIG_MEMCG_V1 - if (!cgroup_subsys_on_dfl(memory_cgrp_subsys)) - return !!memcg->tcpmem_pressure; -#endif /* CONFIG_MEMCG_V1 */ - do { - if (time_before(jiffies, READ_ONCE(memcg->socket_pressure))) - return true; - } while ((memcg =3D parent_mem_cgroup(memcg))); - return false; -} - int alloc_shrinker_info(struct mem_cgroup *memcg); void free_shrinker_info(struct mem_cgroup *memcg); void set_shrinker_bit(struct mem_cgroup *memcg, int nid, int shrinker_id); @@ -1649,11 +1636,6 @@ static inline void mem_cgroup_sk_uncharge(const stru= ct sock *sk, { } =20 -static inline bool mem_cgroup_under_socket_pressure(struct mem_cgroup *mem= cg) -{ - return false; -} - static inline void set_shrinker_bit(struct mem_cgroup *memcg, int nid, int shrinker_id) { diff --git a/include/net/proto_memory.h b/include/net/proto_memory.h index 859e63de81c49..8e91a8fa31b52 100644 --- a/include/net/proto_memory.h +++ b/include/net/proto_memory.h @@ -32,7 +32,7 @@ static inline bool sk_under_memory_pressure(const struct = sock *sk) return false; =20 if (mem_cgroup_sk_enabled(sk) && - mem_cgroup_under_socket_pressure(sk->sk_memcg)) + mem_cgroup_sk_under_memory_pressure(sk)) return true; =20 return !!READ_ONCE(*sk->sk_prot->memory_pressure); diff --git a/include/net/sock.h b/include/net/sock.h index 3efdf680401dd..efb2f659236d4 100644 --- a/include/net/sock.h +++ b/include/net/sock.h @@ -2604,6 +2604,22 @@ static inline bool mem_cgroup_sk_enabled(const struc= t sock *sk) { return mem_cgroup_sockets_enabled && mem_cgroup_from_sk(sk); } + +static inline bool mem_cgroup_sk_under_memory_pressure(const struct sock *= sk) +{ + struct mem_cgroup *memcg =3D mem_cgroup_from_sk(sk); + +#ifdef CONFIG_MEMCG_V1 + if (!cgroup_subsys_on_dfl(memory_cgrp_subsys)) + return !!memcg->tcpmem_pressure; +#endif /* CONFIG_MEMCG_V1 */ + do { + if (time_before(jiffies, READ_ONCE(memcg->socket_pressure))) + return true; + } while ((memcg =3D parent_mem_cgroup(memcg))); + + return false; +} #else static inline struct mem_cgroup *mem_cgroup_from_sk(const struct sock *sk) { @@ -2614,6 +2630,11 @@ static inline bool mem_cgroup_sk_enabled(const struc= t sock *sk) { return false; } + +static inline bool mem_cgroup_sk_under_memory_pressure(const struct sock *= sk) +{ + return false; +} #endif =20 static inline long sock_rcvtimeo(const struct sock *sk, bool noblock) diff --git a/include/net/tcp.h b/include/net/tcp.h index f9a0eb242e65c..9ffe971a1856b 100644 --- a/include/net/tcp.h +++ b/include/net/tcp.h @@ -276,7 +276,7 @@ extern unsigned long tcp_memory_pressure; static inline bool tcp_under_memory_pressure(const struct sock *sk) { if (mem_cgroup_sk_enabled(sk) && - mem_cgroup_under_socket_pressure(sk->sk_memcg)) + mem_cgroup_sk_under_memory_pressure(sk)) return true; =20 return READ_ONCE(tcp_memory_pressure); --=20 2.50.0.727.gbf7dc18ff4-goog From nobody Wed Sep 17 18:19:40 2025 Received: from mail-pg1-f202.google.com (mail-pg1-f202.google.com [209.85.215.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6932F2417EF for ; Mon, 21 Jul 2025 20:36:42 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.215.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1753130203; cv=none; b=BtqrbHPjQTe8uPjasEIKJ+8QoVvstGaXVwbNY3qPnOs0QfYufPpIeECpcTEtF5bxp/DCZ4A/2O28Mw4qkPRLYP1nW9Z5+VJV973uipX0vIo54ZEh9fd37TKbxza889UluiDyRLv3joWw5122B4TpdAJEXdDnbjdaPjgvvqBaWVs= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1753130203; c=relaxed/simple; bh=i5GAzC5L1kuCCltZ38C9FzfkiUrjLYUlG5NNBN2V574=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=QKNFNb8qgLPBt0oWwfcfP7kFLj6Izy3ixAY8TtIrsy3HgnwlQmQcgY84FDjxq5gZciHJ50QTr36lWWCTG1S54EBI/2Dh+50KiX5ueN9eFtsaBRhosoo3ak/YDV28JxQohfh2zZEA9miBjvh9Py/jdAjoz0+EgtkoSeBUV95e/5Y= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--kuniyu.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=va5gvQuC; arc=none smtp.client-ip=209.85.215.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--kuniyu.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="va5gvQuC" Received: by mail-pg1-f202.google.com with SMTP id 41be03b00d2f7-b2c36951518so5326843a12.2 for ; Mon, 21 Jul 2025 13:36:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1753130202; x=1753735002; darn=lists.linux.dev; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=ssPdZm4XAVciTYPYdUcIXie5OXrS2r8M4cSNBMi+lG4=; b=va5gvQuCUU8PxZlOn3XdGLcjggsdeByV8H5o9r7hcPfGoHuSFiqZl92uiaf43siawb 0L1hVIyRu+nogYFDUcUPqtaofnoEAQSJ6bgJbrhxfF21sQLr7IODDRe3E1k94DlZ46lD EdsXDBPdbuvvORmoUyu0MVtjvDSH+CUC5Lbmkd+wToD87+iGrKQ8XCSfwhcmOk26uIZU 1RKNWFD6Eyy9AKTN9oosB5D66PsHNTZgh6WtYlkfiKvtzIAUYoyd25ZoRdhTEoWeHJzg kRdB9HI0QiFiQV9TZXQicSRMs2gyMiYFwoZ21y158slT/HbZtSoamUbS/WfnmRco+7q1 LZ9Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1753130202; x=1753735002; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=ssPdZm4XAVciTYPYdUcIXie5OXrS2r8M4cSNBMi+lG4=; b=YTwmxXQy30gNkz4Ui3/Tonuu3hO8vOgWXHcdVka7UFADcvlLp8qvX9JXn8HXn3XHom snOA6VhAaVkcT0uMelTgb1lqvCxUx6I+5z1KvS55TEgx9nYunGYOhwXOliB+KtSLFvvp 1I7Slsy/liODLI0RmL8HDvwAYayiLg6mR01iwRVhag0FA+z8xPLw92BrF8yxoNONEsXl +ZzoXFJhb+80i3nRq4+unIH7aioSFTAxFayn0NaOryIrdbEtlTrN6Ic5CpxtD8yqSFS1 pW5UvQkmxy1sayGdyNE9Pr08hKWDy8TUU52FcOM6wjaFgew3M8Tdh5hl224D9yD61IKG 0GPQ== X-Forwarded-Encrypted: i=1; AJvYcCXXxj0lRDNpiX8MoPzLRlwI/SX/LdmgyJSGXcezRB5WDac2dyPyJlRT8DWLkS9jyNdMQ4kavg==@lists.linux.dev X-Gm-Message-State: AOJu0Yxz3/Ee5dh0YrHtc3SdZhIluPsLQOUUm6tYGZQae6dIyvFnjgtv Y26Lp86PM+lfGJmyJeVovOlmAw9JF4xlQFwdMsLfT7m5Vqix0NjG2PqaErRtB/j/vkP4l8uK4Gk D4J3smQ== X-Google-Smtp-Source: AGHT+IH0bPuHNRYQE3IBCaVT7B1b82DZOEQMUavwyW5ZrY7G02YZ/hUnE3IIWK5LKSPidRIy8Lw1sZSx8D4= X-Received: from pfblj15.prod.google.com ([2002:a05:6a00:71cf:b0:746:18ec:d11a]) (user=kuniyu job=prod-delivery.src-stubby-dispatcher) by 2002:a05:6a20:9188:b0:233:f0c6:a8a4 with SMTP id adf61e73a8af0-2390dc51bc4mr34981014637.31.1753130201882; Mon, 21 Jul 2025 13:36:41 -0700 (PDT) Date: Mon, 21 Jul 2025 20:35:29 +0000 In-Reply-To: <20250721203624.3807041-1-kuniyu@google.com> Precedence: bulk X-Mailing-List: mptcp@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250721203624.3807041-1-kuniyu@google.com> X-Mailer: git-send-email 2.50.0.727.gbf7dc18ff4-goog Message-ID: <20250721203624.3807041-11-kuniyu@google.com> Subject: [PATCH v1 net-next 10/13] net: Define sk_memcg under CONFIG_MEMCG. From: Kuniyuki Iwashima To: "David S. Miller" , Eric Dumazet , Jakub Kicinski , Neal Cardwell , Paolo Abeni , Willem de Bruijn , Matthieu Baerts , Mat Martineau , Johannes Weiner , Michal Hocko , Roman Gushchin , Shakeel Butt , Andrew Morton Cc: Simon Horman , Geliang Tang , Muchun Song , Kuniyuki Iwashima , Kuniyuki Iwashima , netdev@vger.kernel.org, mptcp@lists.linux.dev, cgroups@vger.kernel.org, linux-mm@kvack.org Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Except for sk_clone_lock(), all accesses to sk->sk_memcg is done under CONFIG_MEMCG. As a bonus, let's define sk->sk_memcg under CONFIG_MEMCG. Signed-off-by: Kuniyuki Iwashima Reviewed-by: Eric Dumazet --- include/net/sock.h | 2 ++ net/core/sock.c | 4 ++++ 2 files changed, 6 insertions(+) diff --git a/include/net/sock.h b/include/net/sock.h index efb2f659236d4..16fe0e5afc587 100644 --- a/include/net/sock.h +++ b/include/net/sock.h @@ -443,7 +443,9 @@ struct sock { __cacheline_group_begin(sock_read_rxtx); int sk_err; struct socket *sk_socket; +#ifdef CONFIG_MEMCG struct mem_cgroup *sk_memcg; +#endif #ifdef CONFIG_XFRM struct xfrm_policy __rcu *sk_policy[2]; #endif diff --git a/net/core/sock.c b/net/core/sock.c index 5537ca2638588..ab6953d295dfa 100644 --- a/net/core/sock.c +++ b/net/core/sock.c @@ -2512,8 +2512,10 @@ struct sock *sk_clone_lock(const struct sock *sk, co= nst gfp_t priority) =20 sock_reset_flag(newsk, SOCK_DONE); =20 +#ifdef CONFIG_MEMCG /* sk->sk_memcg will be populated at accept() time */ newsk->sk_memcg =3D NULL; +#endif =20 cgroup_sk_clone(&newsk->sk_cgrp_data); =20 @@ -4452,7 +4454,9 @@ static int __init sock_struct_check(void) =20 CACHELINE_ASSERT_GROUP_MEMBER(struct sock, sock_read_rxtx, sk_err); CACHELINE_ASSERT_GROUP_MEMBER(struct sock, sock_read_rxtx, sk_socket); +#ifdef CONFIG_MEMCG CACHELINE_ASSERT_GROUP_MEMBER(struct sock, sock_read_rxtx, sk_memcg); +#endif =20 CACHELINE_ASSERT_GROUP_MEMBER(struct sock, sock_write_rxtx, sk_lock); CACHELINE_ASSERT_GROUP_MEMBER(struct sock, sock_write_rxtx, sk_reserved_m= em); --=20 2.50.0.727.gbf7dc18ff4-goog From nobody Wed Sep 17 18:19:40 2025 Received: from mail-pg1-f202.google.com (mail-pg1-f202.google.com [209.85.215.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 4ADD4242925 for ; Mon, 21 Jul 2025 20:36:43 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.215.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1753130205; cv=none; b=eyogiHrlbTX2+qNyKs066HQqRsz1hAsDYL/s4Eal8YGhu/9q1hmZm1QjcnHjWeD05Kj7B6VJdPMWGxcePrPo+0SnIryLWVWwLqpvHwfShG/eQ05z4O5/q2nITa9PNMp3rkdRpRCsolZQV9lR4nz3Id8GfwYWuWohJHo1kGXDctU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1753130205; c=relaxed/simple; bh=i+zfIWSk5Yyh+CsgJiQoi9JopyK74XFsgZ0y4qXuiVE=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=XrludAxXUNasRErvcc/NoaHotAVi5OHlvtQORdBGUhwSeUeiY+17g/Y4wHxIlB4JerCLFruP2aeqWJvQ2bhhzKE5pia3szYy5xI0z8U76mt8ACGTBaxSun5A1fSeiOIl8ftgnLf2dla2VmxxFvgWjBDv0XllQVDZYT16dmMNNCs= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--kuniyu.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=gCeZl1It; arc=none smtp.client-ip=209.85.215.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--kuniyu.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="gCeZl1It" Received: by mail-pg1-f202.google.com with SMTP id 41be03b00d2f7-b39280167fdso3400442a12.0 for ; Mon, 21 Jul 2025 13:36:43 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1753130203; x=1753735003; darn=lists.linux.dev; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=gOvDOtNuHuKEQ0AG8licv0sCiHFkG4hjeCZ8hfYKZ/g=; b=gCeZl1ItfO+BcBDmdtcCeaXa68e5KfbmXh3BDeaGS9egtVF7JNhRLE3K+UGUjLRGPF ljma7uQ7j0n+UKf5dk6eOs/Glkmq4IRrY+VmA0U6SGAu/+GckqZPs8i8oEGKjvNI4xoF BudpJqKNMQc+tAGFQJv01ImlO+C1DkU+M/VsF2lxH7EkhK0puO4XAxzwjl+ILb/omKBY E+T9DnjhxGChTBQTVbcCE9dsQDrZjobqsm9dmu0hMkISATK/v8ZRXozBj5TrbaljDtea KO0g2QYGX9OSWLLd/InJVkUqWZSeyYNLsZjcpOJjOnoXcimPU1T70tubA2lT35+UaorK olfw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1753130203; x=1753735003; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=gOvDOtNuHuKEQ0AG8licv0sCiHFkG4hjeCZ8hfYKZ/g=; b=GFuzsgIXDvK8o0+RnpQKQIsBPn82Dke72axK1hJZZTSAHcWnnWuuG8srQSLhpifZ6r e4lBN4yMThYZwrNGYmxUGe0Jr8hyK90dh1QM4J/myfdpwgzksXD56zWURagM3kGZSyLu XdG61ve/uHHV2zDINWjIp4vTvDgMAVA3HdvYcWmXECR1c15va6q28C0VFN2CQCtAqe/G P0WNiS4BLZtQZ5OEpS5We4HUJ6PyFHiZsPkXGpjU5EVsp/e7CtgYbyzQ6HYqOlKVSaeO ywcQBCLu6o5g13aJQgS/w5lDiGuwBmabzSxtuoFqB65HJPM+43iZKWdA+igsF+nT7vA7 1TWw== X-Forwarded-Encrypted: i=1; AJvYcCVpP+7iY+mfAhh3e4G/5IwY2xpa3FvIIqj49kulqR1x2Kry3ScF3f1evLKIRkHXQ2e0tWysmw==@lists.linux.dev X-Gm-Message-State: AOJu0YzklBdoLxe/FY8Iwpyas6A/qzuLzNPPUNv9Pq/wIe8SvMsWzSud wG92DiB+K+h+CEhnVG5J7tUrMSFNrKbsIbWaG5+npr4KB/6VvfSj1/BFKMGqw5pPLkFQ2rNEcLU Ro9Q02A== X-Google-Smtp-Source: AGHT+IFVGe379XDWUF+9CY2yjFXvtKB+09OD1v073Z06Drs+S2iB6DNfmLq4LLNVW9Zv/pSwEo/nmwsHGbs= X-Received: from pfbkx24.prod.google.com ([2002:a05:6a00:6f18:b0:747:7188:c30]) (user=kuniyu job=prod-delivery.src-stubby-dispatcher) by 2002:a05:6a20:9150:b0:234:4f62:9b3e with SMTP id adf61e73a8af0-237d701a40cmr37381983637.27.1753130203466; Mon, 21 Jul 2025 13:36:43 -0700 (PDT) Date: Mon, 21 Jul 2025 20:35:30 +0000 In-Reply-To: <20250721203624.3807041-1-kuniyu@google.com> Precedence: bulk X-Mailing-List: mptcp@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250721203624.3807041-1-kuniyu@google.com> X-Mailer: git-send-email 2.50.0.727.gbf7dc18ff4-goog Message-ID: <20250721203624.3807041-12-kuniyu@google.com> Subject: [PATCH v1 net-next 11/13] net-memcg: Add memory.socket_isolated knob. From: Kuniyuki Iwashima To: "David S. Miller" , Eric Dumazet , Jakub Kicinski , Neal Cardwell , Paolo Abeni , Willem de Bruijn , Matthieu Baerts , Mat Martineau , Johannes Weiner , Michal Hocko , Roman Gushchin , Shakeel Butt , Andrew Morton Cc: Simon Horman , Geliang Tang , Muchun Song , Kuniyuki Iwashima , Kuniyuki Iwashima , netdev@vger.kernel.org, mptcp@lists.linux.dev, cgroups@vger.kernel.org, linux-mm@kvack.org Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Some networking protocols have their own global memory accounting, and such memory is also charged to memcg as sock in memory.stat. Such sockets are subject to the global limit, thus affected by a noisy neighbour outside the cgroup. We will decouple the global memory accounting if configured. Let's add a per-memcg knob to control that. The value will be saved in each socket when created and will persist through the socket's lifetime. Signed-off-by: Kuniyuki Iwashima Reviewed-by: Eric Dumazet --- Documentation/admin-guide/cgroup-v2.rst | 16 +++++++++++ include/linux/memcontrol.h | 6 ++++ include/net/sock.h | 3 ++ mm/memcontrol.c | 37 +++++++++++++++++++++++++ 4 files changed, 62 insertions(+) diff --git a/Documentation/admin-guide/cgroup-v2.rst b/Documentation/admin-= guide/cgroup-v2.rst index bd98ea3175ec1..2428707b7d27d 100644 --- a/Documentation/admin-guide/cgroup-v2.rst +++ b/Documentation/admin-guide/cgroup-v2.rst @@ -1878,6 +1878,22 @@ The following nested keys are defined. Shows pressure stall information for memory. See :ref:`Documentation/accounting/psi.rst ` for details. =20 + memory.socket_isolated + A read-write single value file which exists on non-root cgroups. + The default value is "0". + + Some networking protocols (e.g., TCP, UDP) implement their own memory + accounting for socket buffers. + + This memory is also charged to a non-root cgroup as sock in memory.stat. + + Since per-protocol limits such as /proc/sys/net/ipv4/tcp_mem and + /proc/sys/net/ipv4/udp_mem are global, memory allocation for socket + buffers may fail even when the cgroup has available memory. + + Sockets created with socket_isolated set to 1 are no longer subject + to these global protocol limits. + =20 Usage Guidelines ~~~~~~~~~~~~~~~~ diff --git a/include/linux/memcontrol.h b/include/linux/memcontrol.h index 211712ec57d1a..7d5d43e3b49e6 100644 --- a/include/linux/memcontrol.h +++ b/include/linux/memcontrol.h @@ -226,6 +226,12 @@ struct mem_cgroup { */ bool oom_group; =20 + /* + * If set, MEMCG_SOCK memory is charged on memcg only, + * otherwise, memcg and sk->sk_prot->memory_allocated. + */ + bool socket_isolated; + int swappiness; =20 /* memory.events and memory.events.local */ diff --git a/include/net/sock.h b/include/net/sock.h index 16fe0e5afc587..5e8c73731531c 100644 --- a/include/net/sock.h +++ b/include/net/sock.h @@ -2597,6 +2597,9 @@ static inline gfp_t gfp_memcg_charge(void) } =20 #ifdef CONFIG_MEMCG + +#define MEMCG_SOCK_ISOLATED 1UL + static inline struct mem_cgroup *mem_cgroup_from_sk(const struct sock *sk) { return sk->sk_memcg; diff --git a/mm/memcontrol.c b/mm/memcontrol.c index d7f4e31f4e625..0a55c12a6679b 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -4645,6 +4645,37 @@ static ssize_t memory_reclaim(struct kernfs_open_fil= e *of, char *buf, return nbytes; } =20 +static int memory_socket_isolated_show(struct seq_file *m, void *v) +{ + struct mem_cgroup *memcg =3D mem_cgroup_from_seq(m); + + seq_printf(m, "%d\n", READ_ONCE(memcg->socket_isolated)); + + return 0; +} + +static ssize_t memory_socket_isolated_write(struct kernfs_open_file *of, + char *buf, size_t nbytes, loff_t off) +{ + struct mem_cgroup *memcg =3D mem_cgroup_from_css(of_css(of)); + int ret, socket_isolated; + + buf =3D strstrip(buf); + if (!buf) + return -EINVAL; + + ret =3D kstrtoint(buf, 0, &socket_isolated); + if (ret) + return ret; + + if (socket_isolated !=3D 0 && socket_isolated !=3D MEMCG_SOCK_ISOLATED) + return -EINVAL; + + WRITE_ONCE(memcg->socket_isolated, socket_isolated); + + return nbytes; +} + static struct cftype memory_files[] =3D { { .name =3D "current", @@ -4716,6 +4747,12 @@ static struct cftype memory_files[] =3D { .flags =3D CFTYPE_NS_DELEGATABLE, .write =3D memory_reclaim, }, + { + .name =3D "socket_isolated", + .flags =3D CFTYPE_NOT_ON_ROOT, + .seq_show =3D memory_socket_isolated_show, + .write =3D memory_socket_isolated_write, + }, { } /* terminate */ }; =20 --=20 2.50.0.727.gbf7dc18ff4-goog From nobody Wed Sep 17 18:19:40 2025 Received: from mail-pf1-f202.google.com (mail-pf1-f202.google.com [209.85.210.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 67D6823D287 for ; Mon, 21 Jul 2025 20:36:45 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1753130206; cv=none; b=L/3rOPfjZ7A0SkQSD7Vx78dqRdQmeVRl2HJI/+i1HKx1xrOlSSxMtSMjYN0IT8ZfQEBXJHexVKcE/IPe4FDX8lF593D1yoMTmVlN+XU7LtfKgeXr+AD6frNyOpRb2adxwRPkuTXp7nSq/N4Hbdcxcl4Z58aIlvxPm1dBb05N9vw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1753130206; c=relaxed/simple; bh=zcJtco0ccgJTHzB1ejYPUAyCNXBYGZIb0tH+3/ptyMM=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=pR+WW+YWyV+JkOMiFhzx0ZYgvfCJDM53/UaBYwbGcw7XOL1dk4TgJFGmJgu9KBFsUoUpsMvdrC/P3CSsAsMH9n0g69AZ5N+CW1YaDBWKDV9VSVDl0S7gl2bd++nrfE2mVd5cICNvWPpARLpmCbtfkN+UEb4sbpz1OnbK2akyiWg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--kuniyu.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=f9rPAZCh; arc=none smtp.client-ip=209.85.210.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--kuniyu.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="f9rPAZCh" Received: by mail-pf1-f202.google.com with SMTP id d2e1a72fcca58-756bb07b029so4242633b3a.1 for ; Mon, 21 Jul 2025 13:36:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1753130205; x=1753735005; darn=lists.linux.dev; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=xdQJmy4blGtPG7gm97FaKmrzo1iYWfvkpp5t2qCAUbc=; b=f9rPAZCh1hMuxNELo2t7a/vPBsIgcfoRuqgRG2hXOtpCW3OpeQXVY40Wf1CQH+G38G BMslwWjDzW48t5XqIjI1IkcJZ2/ueE9yihvV8cA3VkRwUPHVQfk5Uwca0YFFKFLOF5R7 88yR2VtNuJtCKLGzokUIeD/d70A64NHCmEYj+ndVScOSLPXGtqolO1lvUVLkxdrWhYYa F95Wss6Y6DHekYNDBemohEq2pIQ2S69rKpaw1mMBdmsc88FjKNhkE1E+1Mk/sXbZnb4v tnI5wybm7KcNkqL3uOaDyZNwX53M2cxprVfOZOGwGDcHanpmbQ6EMVZchwLXvx20L3XH lKNw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1753130205; x=1753735005; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=xdQJmy4blGtPG7gm97FaKmrzo1iYWfvkpp5t2qCAUbc=; b=aQzg3YsiDZpAHWsUPPjvi9jsznl19A2AO9WXpg4UfRRqEyrdLv2ndYZO9ipluoJdf/ paNq8UsMhhqaODCnuBxUipWfsbemFNQE8BVvzgHMndA5PJbZwl3kvZIxIgnXNeogNQWd qM+kv1tGif5TBFM4+or7AtZ1ri2KQJMp8gz9TkBhaEG3HjENd1I3P5fmBu9xy9755wSD 5SiKbJUkJAYhFbgxu8kHpAVYC9PaqzMtIDaKGSvfLgw9TFhIrsdtrWSwgBhwOXt5lBjG HM3wKwcUeHwJqZgOBE89/omIbmSKe7JxyQQNAUQrr3STjvVEKSrLuYG/1kbxNwveAnX7 LFxg== X-Forwarded-Encrypted: i=1; AJvYcCXxWSIvmgMXab8eGhc78W+yZWu6Cry0nRE+fFxaJtjQWusvGWWthOpxHcDtXJAAYrwMCDhmIg==@lists.linux.dev X-Gm-Message-State: AOJu0YzfhbRyk+ObqTUFL/wCbCuBwSrJtWC36ry9zoZfz3cEO6lcjsEh AdOi/7MyAn0clUQHQ7fyj5Uj3ffM6Nfkvt9IK81vVBHuD5c5WHm9A12HTxOF0XU1TbuQKvzc2kV J5pJoQA== X-Google-Smtp-Source: AGHT+IH3cVl1mpYtpoQ+O+03muSwme2Q/Ci4haqA3DdzxtA/Bzy9F6BXAV1IYuYuGpbybBp0dJL0GcI6AFY= X-Received: from pfoo21.prod.google.com ([2002:a05:6a00:1a15:b0:747:a97f:513f]) (user=kuniyu job=prod-delivery.src-stubby-dispatcher) by 2002:a05:6a00:3c88:b0:742:a77b:8c4 with SMTP id d2e1a72fcca58-7572267b30cmr34198928b3a.3.1753130204945; Mon, 21 Jul 2025 13:36:44 -0700 (PDT) Date: Mon, 21 Jul 2025 20:35:31 +0000 In-Reply-To: <20250721203624.3807041-1-kuniyu@google.com> Precedence: bulk X-Mailing-List: mptcp@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250721203624.3807041-1-kuniyu@google.com> X-Mailer: git-send-email 2.50.0.727.gbf7dc18ff4-goog Message-ID: <20250721203624.3807041-13-kuniyu@google.com> Subject: [PATCH v1 net-next 12/13] net-memcg: Store memcg->socket_isolated in sk->sk_memcg. From: Kuniyuki Iwashima To: "David S. Miller" , Eric Dumazet , Jakub Kicinski , Neal Cardwell , Paolo Abeni , Willem de Bruijn , Matthieu Baerts , Mat Martineau , Johannes Weiner , Michal Hocko , Roman Gushchin , Shakeel Butt , Andrew Morton Cc: Simon Horman , Geliang Tang , Muchun Song , Kuniyuki Iwashima , Kuniyuki Iwashima , netdev@vger.kernel.org, mptcp@lists.linux.dev, cgroups@vger.kernel.org, linux-mm@kvack.org Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" memcg->socket_isolated can change at any time, so we must snapshot the value for each socket to ensure consistency. Given sk->sk_memcg can be accessed in the fast path, it would be preferable to place the flag field in the same cache line as sk->sk_memcg. However, struct sock does not have such a 1-byte hole. Let's store the flag in the lowest bit of sk->sk_memcg and add a helper to check the bit. Signed-off-by: Kuniyuki Iwashima Reviewed-by: Eric Dumazet --- include/net/sock.h | 20 +++++++++++++++++++- mm/memcontrol.c | 13 +++++++++++-- 2 files changed, 30 insertions(+), 3 deletions(-) diff --git a/include/net/sock.h b/include/net/sock.h index 5e8c73731531c..2e9d76fc2bf38 100644 --- a/include/net/sock.h +++ b/include/net/sock.h @@ -2599,10 +2599,16 @@ static inline gfp_t gfp_memcg_charge(void) #ifdef CONFIG_MEMCG =20 #define MEMCG_SOCK_ISOLATED 1UL +#define MEMCG_SOCK_FLAG_MASK MEMCG_SOCK_ISOLATED +#define MEMCG_SOCK_PTR_MASK ~(MEMCG_SOCK_FLAG_MASK) =20 static inline struct mem_cgroup *mem_cgroup_from_sk(const struct sock *sk) { - return sk->sk_memcg; + unsigned long val =3D (unsigned long)sk->sk_memcg; + + val &=3D MEMCG_SOCK_PTR_MASK; + + return (struct mem_cgroup *)val; } =20 static inline bool mem_cgroup_sk_enabled(const struct sock *sk) @@ -2610,6 +2616,13 @@ static inline bool mem_cgroup_sk_enabled(const struc= t sock *sk) return mem_cgroup_sockets_enabled && mem_cgroup_from_sk(sk); } =20 +static inline bool mem_cgroup_sk_isolated(const struct sock *sk) +{ + struct mem_cgroup *memcg =3D sk->sk_memcg; + + return (unsigned long)memcg & MEMCG_SOCK_ISOLATED; +} + static inline bool mem_cgroup_sk_under_memory_pressure(const struct sock *= sk) { struct mem_cgroup *memcg =3D mem_cgroup_from_sk(sk); @@ -2636,6 +2649,11 @@ static inline bool mem_cgroup_sk_enabled(const struc= t sock *sk) return false; } =20 +static inline bool mem_cgroup_sk_isolated(const struct sock *sk) +{ + return false; +} + static inline bool mem_cgroup_sk_under_memory_pressure(const struct sock *= sk) { return false; diff --git a/mm/memcontrol.c b/mm/memcontrol.c index 0a55c12a6679b..85decc4319f96 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -5098,6 +5098,15 @@ void mem_cgroup_migrate(struct folio *old, struct fo= lio *new) DEFINE_STATIC_KEY_FALSE(memcg_sockets_enabled_key); EXPORT_SYMBOL(memcg_sockets_enabled_key); =20 +static void mem_cgroup_sk_set(struct sock *sk, const struct mem_cgroup *me= mcg) +{ + unsigned long val =3D (unsigned long)memcg; + + val |=3D READ_ONCE(memcg->socket_isolated); + + sk->sk_memcg =3D (struct mem_cgroup *)val; +} + void mem_cgroup_sk_alloc(struct sock *sk) { struct mem_cgroup *memcg; @@ -5116,7 +5125,7 @@ void mem_cgroup_sk_alloc(struct sock *sk) if (!cgroup_subsys_on_dfl(memory_cgrp_subsys) && !memcg1_tcpmem_active(me= mcg)) goto out; if (css_tryget(&memcg->css)) - sk->sk_memcg =3D memcg; + mem_cgroup_sk_set(sk, memcg); out: rcu_read_unlock(); } @@ -5138,7 +5147,7 @@ void mem_cgroup_sk_inherit(const struct sock *sk, str= uct sock *newsk) =20 mem_cgroup_sk_free(newsk); css_get(&memcg->css); - newsk->sk_memcg =3D memcg; + mem_cgroup_sk_set(newsk, memcg); } =20 /** --=20 2.50.0.727.gbf7dc18ff4-goog From nobody Wed Sep 17 18:19:40 2025 Received: from mail-pf1-f202.google.com (mail-pf1-f202.google.com [209.85.210.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id EFF9F24500A for ; Mon, 21 Jul 2025 20:36:46 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1753130209; cv=none; b=TaVOkZqQSIqK+NWpM2o0SfOyL797GAxA9hNqUzKVz472HG+i5OU97OL5D/yxLelmkEL6ZPlofttN8JD+KZQJBr6jBb8ICddOTDv0h2dvafT/bhZ8fkR+YeICfp60SULT414Q9tSe3ZUqhBZbThbepnSQyqNAoJlJCiEW3IWRsCI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1753130209; c=relaxed/simple; bh=PASGG2Ff90fjH/vqEYJ89hhiHfQMtaYoG/eC4+HIyh4=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=bvvZ479XiU6SFWXkRIiodrJ/1gsCN7UVltJgE7Tv30m8UcH2QKayQrEBmw1nC5777RyQqvadL98yqbdnxdEwEJLNmg5N1opcx1yC9qfa3Wsj1dERxaRpPrwe7VdIAc3VQ9Zh+4Ewvem9MEzNX0a4W27K8MJRq376IzaZ77EKwn0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--kuniyu.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=So60jI/q; arc=none smtp.client-ip=209.85.210.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--kuniyu.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="So60jI/q" Received: by mail-pf1-f202.google.com with SMTP id d2e1a72fcca58-740774348f6so4365768b3a.1 for ; Mon, 21 Jul 2025 13:36:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1753130206; x=1753735006; darn=lists.linux.dev; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=nUinzOTBmFzGlPRyqzH40paksGVV4+5eLWcmsRYDtpk=; b=So60jI/qKvhPxDNwU+dmMIt0zLSXWxDtm88ugsYEF9xgLkKM6SojNgFwlWeJJVaFjz L9gHZs8kP2P7o6hfh8IeYgAmx7Kqv7chazi1xFSv2ZGleVQNy9LKf0DFu/rZpyCJQZso ofsjbmbrkLQhDoRMZHNxj3ryKOESB5E1IzmbyNGtq8hzl6o7MOvvhBB7Gp70oaGEMwmc 9Lqboj79/3SYZe3nUwRS0qph1g727Oa+fINbL0POmwWKRpjIBN9TlqONBC0IK5skX12f J2gCPEhLDNoIUe9YXfKHXUN6NsuJpZqsfXtxWpykqZo/7shZra7kl0d+nojDCTksmZm4 I3MQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1753130206; x=1753735006; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=nUinzOTBmFzGlPRyqzH40paksGVV4+5eLWcmsRYDtpk=; b=MObNNIBOn8YGf7fZXm1Q/IomOo8QHZk2/XVF3xmTvTz9EIeQ5PGvRVrutK8SIQi0FZ /H2Ik64YK9PW2aG4uMpKmkG++wQxWmYK1Z5bJ/ZiMbhhUurNsF3X1Juh9CHC81Q/iIqU V2zyYHlVhPO+YXe5DJInPl3d8IlNzEsvHTuuKLCdrQ7n3gK4wY94NF2hS3GmzHVYAecO YUASvMUviJwwyS15nShXbmraq8xB6D3pTbcJ+wppdj4vZ1z6VPFr53J8Jz4RBJBscK49 Zpsww7BQmSaElocQ7t4X+H1BXuJc7dv/p8gA0ghjCVHg4ffWnAwExHHpD8oZop/OY/3u GOeQ== X-Forwarded-Encrypted: i=1; AJvYcCWjOEuRxTYff4NFNRzB+lAyRC/KPjuJ0+RrGYGtNjkiWeXr5PE9nlY+naHWGs9ul/UaSuz9lw==@lists.linux.dev X-Gm-Message-State: AOJu0Yzgpg+DcSfsiMFybQr1Im3DLv+XfFbaPC1W5rLnYJ1jCklLNVwq zSqhb+wEnKT2EXZSfJl9NaEXJJ34yHEsPXijjBrJkrZdzjrYunrilcvscqq7b7ZeLNMfFh2gsOq x2gEPbg== X-Google-Smtp-Source: AGHT+IF55Tasmzf40vI7WdvTmtciM9z/zourqwpEu8s96++APVGs+R76gSIW5Yf92/sZUKZ1BwWIx+Puq2c= X-Received: from pfoo15.prod.google.com ([2002:a05:6a00:1a0f:b0:746:683a:6104]) (user=kuniyu job=prod-delivery.src-stubby-dispatcher) by 2002:a05:6a20:3d1a:b0:230:f120:f7f3 with SMTP id adf61e73a8af0-2391c92c40bmr23246374637.8.1753130206377; Mon, 21 Jul 2025 13:36:46 -0700 (PDT) Date: Mon, 21 Jul 2025 20:35:32 +0000 In-Reply-To: <20250721203624.3807041-1-kuniyu@google.com> Precedence: bulk X-Mailing-List: mptcp@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250721203624.3807041-1-kuniyu@google.com> X-Mailer: git-send-email 2.50.0.727.gbf7dc18ff4-goog Message-ID: <20250721203624.3807041-14-kuniyu@google.com> Subject: [PATCH v1 net-next 13/13] net-memcg: Allow decoupling memcg from global protocol memory accounting. From: Kuniyuki Iwashima To: "David S. Miller" , Eric Dumazet , Jakub Kicinski , Neal Cardwell , Paolo Abeni , Willem de Bruijn , Matthieu Baerts , Mat Martineau , Johannes Weiner , Michal Hocko , Roman Gushchin , Shakeel Butt , Andrew Morton Cc: Simon Horman , Geliang Tang , Muchun Song , Kuniyuki Iwashima , Kuniyuki Iwashima , netdev@vger.kernel.org, mptcp@lists.linux.dev, cgroups@vger.kernel.org, linux-mm@kvack.org Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Some protocols (e.g., TCP, UDP) implement memory accounting for socket buffers and charge memory to per-protocol global counters pointed to by sk->sk_proto->memory_allocated. When running under a non-root cgroup, this memory is also charged to the memcg as sock in memory.stat. Even when memory usage is controlled by memcg, sockets using such protocols are still subject to global limits (e.g., /proc/sys/net/ipv4/tcp_mem). This makes it difficult to accurately estimate and configure appropriate global limits, especially in multi-tenant environments. If all workloads were guaranteed to be controlled under memcg, the issue could be worked around by setting tcp_mem[0~2] to UINT_MAX. In reality, this assumption does not always hold, and a single workload that opts out of memcg can consume memory up to the global limit, becoming a noisy neighbour. Let's decouple memcg from the global per-protocol memory accounting. This simplifies memcg configuration while keeping the global limits within a reasonable range. If mem_cgroup_sk_isolated(sk) returns true, the per-protocol memory accounting is skipped. In inet_csk_accept(), we need to reclaim counts that are already charged for child sockets because we do not allocate sk->sk_memcg until accept(). Note that trace_sock_exceed_buf_limit() will always show 0 as accounted for the isolated sockets, but this can be obtained via memory.stat. Tested with a script that creates local socket pairs and send()s a bunch of data without recv()ing. Setup: # mkdir /sys/fs/cgroup/test # echo $$ >> /sys/fs/cgroup/test/cgroup.procs # sysctl -q net.ipv4.tcp_mem=3D"1000 1000 1000" Without memory.socket_isolated: # echo 0 > /sys/fs/cgroup/test/memory.socket_isolated # prlimit -n=3D524288:524288 bash -c "python3 pressure.py" & # cat /sys/fs/cgroup/test/memory.stat | grep sock sock 24682496 # ss -tn | head -n 5 State Recv-Q Send-Q Local Address:Port Peer Address:Port ESTAB 2000 0 127.0.0.1:54997 127.0.0.1:37738 ESTAB 2000 0 127.0.0.1:54997 127.0.0.1:60122 ESTAB 2000 0 127.0.0.1:54997 127.0.0.1:33622 ESTAB 2000 0 127.0.0.1:54997 127.0.0.1:35042 # nstat | grep Pressure || echo no pressure TcpExtTCPMemoryPressures 1 0.0 With memory.socket_isolated: # echo 1 > /sys/fs/cgroup/test/memory.socket_isolated # prlimit -n=3D524288:524288 bash -c "python3 pressure.py" & # cat /sys/fs/cgroup/test/memory.stat | grep sock sock 2766671872 # ss -tn | head -n 5 State Recv-Q Send-Q Local Address:Port Peer Address:Port ESTAB 112000 0 127.0.0.1:41729 127.0.0.1:35062 ESTAB 110000 0 127.0.0.1:41729 127.0.0.1:36288 ESTAB 112000 0 127.0.0.1:41729 127.0.0.1:37560 ESTAB 112000 0 127.0.0.1:41729 127.0.0.1:37096 # nstat | grep Pressure || echo no pressure no pressure Signed-off-by: Kuniyuki Iwashima --- include/net/proto_memory.h | 10 +++-- include/net/tcp.h | 10 +++-- net/core/sock.c | 65 +++++++++++++++++++++++---------- net/ipv4/inet_connection_sock.c | 18 +++++++-- net/ipv4/tcp_output.c | 10 ++++- 5 files changed, 82 insertions(+), 31 deletions(-) diff --git a/include/net/proto_memory.h b/include/net/proto_memory.h index 8e91a8fa31b52..3c2e92f5a6866 100644 --- a/include/net/proto_memory.h +++ b/include/net/proto_memory.h @@ -31,9 +31,13 @@ static inline bool sk_under_memory_pressure(const struct= sock *sk) if (!sk->sk_prot->memory_pressure) return false; =20 - if (mem_cgroup_sk_enabled(sk) && - mem_cgroup_sk_under_memory_pressure(sk)) - return true; + if (mem_cgroup_sk_enabled(sk)) { + if (mem_cgroup_sk_under_memory_pressure(sk)) + return true; + + if (mem_cgroup_sk_isolated(sk)) + return false; + } =20 return !!READ_ONCE(*sk->sk_prot->memory_pressure); } diff --git a/include/net/tcp.h b/include/net/tcp.h index 9ffe971a1856b..a5ff82a59867b 100644 --- a/include/net/tcp.h +++ b/include/net/tcp.h @@ -275,9 +275,13 @@ extern unsigned long tcp_memory_pressure; /* optimized version of sk_under_memory_pressure() for TCP sockets */ static inline bool tcp_under_memory_pressure(const struct sock *sk) { - if (mem_cgroup_sk_enabled(sk) && - mem_cgroup_sk_under_memory_pressure(sk)) - return true; + if (mem_cgroup_sk_enabled(sk)) { + if (mem_cgroup_sk_under_memory_pressure(sk)) + return true; + + if (mem_cgroup_sk_isolated(sk)) + return false; + } =20 return READ_ONCE(tcp_memory_pressure); } diff --git a/net/core/sock.c b/net/core/sock.c index ab6953d295dfa..e1ae6d03b8227 100644 --- a/net/core/sock.c +++ b/net/core/sock.c @@ -1046,17 +1046,21 @@ static int sock_reserve_memory(struct sock *sk, int= bytes) if (!charged) return -ENOMEM; =20 - /* pre-charge to forward_alloc */ - sk_memory_allocated_add(sk, pages); - allocated =3D sk_memory_allocated(sk); - /* If the system goes into memory pressure with this - * precharge, give up and return error. - */ - if (allocated > sk_prot_mem_limits(sk, 1)) { - sk_memory_allocated_sub(sk, pages); - mem_cgroup_sk_uncharge(sk, pages); - return -ENOMEM; + if (!mem_cgroup_sk_isolated(sk)) { + /* pre-charge to forward_alloc */ + sk_memory_allocated_add(sk, pages); + allocated =3D sk_memory_allocated(sk); + + /* If the system goes into memory pressure with this + * precharge, give up and return error. + */ + if (allocated > sk_prot_mem_limits(sk, 1)) { + sk_memory_allocated_sub(sk, pages); + mem_cgroup_sk_uncharge(sk, pages); + return -ENOMEM; + } } + sk_forward_alloc_add(sk, pages << PAGE_SHIFT); =20 WRITE_ONCE(sk->sk_reserved_mem, @@ -3153,8 +3157,12 @@ bool sk_page_frag_refill(struct sock *sk, struct pag= e_frag *pfrag) if (likely(skb_page_frag_refill(32U, pfrag, sk->sk_allocation))) return true; =20 - sk_enter_memory_pressure(sk); sk_stream_moderate_sndbuf(sk); + + if (mem_cgroup_sk_enabled(sk) && mem_cgroup_sk_isolated(sk)) + return false; + + sk_enter_memory_pressure(sk); return false; } EXPORT_SYMBOL(sk_page_frag_refill); @@ -3267,18 +3275,30 @@ int __sk_mem_raise_allocated(struct sock *sk, int s= ize, int amt, int kind) { bool memcg_enabled =3D false, charged =3D false; struct proto *prot =3D sk->sk_prot; - long allocated; - - sk_memory_allocated_add(sk, amt); - allocated =3D sk_memory_allocated(sk); + long allocated =3D 0; =20 if (mem_cgroup_sk_enabled(sk)) { + bool isolated =3D mem_cgroup_sk_isolated(sk); + memcg_enabled =3D true; charged =3D mem_cgroup_sk_charge(sk, amt, gfp_memcg_charge()); - if (!charged) + + if (isolated && charged) + return 1; + + if (!charged) { + if (!isolated) { + sk_memory_allocated_add(sk, amt); + allocated =3D sk_memory_allocated(sk); + } + goto suppress_allocation; + } } =20 + sk_memory_allocated_add(sk, amt); + allocated =3D sk_memory_allocated(sk); + /* Under limit. */ if (allocated <=3D sk_prot_mem_limits(sk, 0)) { sk_leave_memory_pressure(sk); @@ -3357,7 +3377,8 @@ int __sk_mem_raise_allocated(struct sock *sk, int siz= e, int amt, int kind) =20 trace_sock_exceed_buf_limit(sk, prot, allocated, kind); =20 - sk_memory_allocated_sub(sk, amt); + if (allocated) + sk_memory_allocated_sub(sk, amt); =20 if (charged) mem_cgroup_sk_uncharge(sk, amt); @@ -3396,11 +3417,15 @@ EXPORT_SYMBOL(__sk_mem_schedule); */ void __sk_mem_reduce_allocated(struct sock *sk, int amount) { - sk_memory_allocated_sub(sk, amount); - - if (mem_cgroup_sk_enabled(sk)) + if (mem_cgroup_sk_enabled(sk)) { mem_cgroup_sk_uncharge(sk, amount); =20 + if (mem_cgroup_sk_isolated(sk)) + return; + } + + sk_memory_allocated_sub(sk, amount); + if (sk_under_global_memory_pressure(sk) && (sk_memory_allocated(sk) < sk_prot_mem_limits(sk, 0))) sk_leave_memory_pressure(sk); diff --git a/net/ipv4/inet_connection_sock.c b/net/ipv4/inet_connection_soc= k.c index 0ef1eacd539d1..9d56085f7f54b 100644 --- a/net/ipv4/inet_connection_sock.c +++ b/net/ipv4/inet_connection_sock.c @@ -22,6 +22,7 @@ #include #include #include +#include =20 #if IS_ENABLED(CONFIG_IPV6) /* match_sk*_wildcard =3D=3D true: IPV6_ADDR_ANY equals to any IPv6 addre= sses @@ -710,7 +711,6 @@ struct sock *inet_csk_accept(struct sock *sk, struct pr= oto_accept_arg *arg) =20 if (mem_cgroup_sockets_enabled) { gfp_t gfp =3D GFP_KERNEL | __GFP_NOFAIL; - int amt =3D 0; =20 /* atomically get the memory usage, set and charge the * newsk->sk_memcg. @@ -719,15 +719,27 @@ struct sock *inet_csk_accept(struct sock *sk, struct = proto_accept_arg *arg) =20 mem_cgroup_sk_alloc(newsk); if (mem_cgroup_from_sk(newsk)) { + int amt; + /* The socket has not been accepted yet, no need * to look at newsk->sk_wmem_queued. */ amt =3D sk_mem_pages(newsk->sk_forward_alloc + atomic_read(&newsk->sk_rmem_alloc)); + if (amt) { + /* This amt is already charged globally to + * sk_prot->memory_allocated due to lack of + * sk_memcg until accept(), thus we need to + * reclaim it here if newsk is isolated. + */ + if (mem_cgroup_sk_isolated(newsk)) + sk_memory_allocated_sub(newsk, amt); + + mem_cgroup_sk_charge(newsk, amt, gfp); + } + } =20 - if (amt) - mem_cgroup_sk_charge(newsk, amt, gfp); kmem_cache_charge(newsk, gfp); =20 release_sock(newsk); diff --git a/net/ipv4/tcp_output.c b/net/ipv4/tcp_output.c index 09f0802f36afa..79e705fca8b67 100644 --- a/net/ipv4/tcp_output.c +++ b/net/ipv4/tcp_output.c @@ -3562,12 +3562,18 @@ void sk_forced_mem_schedule(struct sock *sk, int si= ze) delta =3D size - sk->sk_forward_alloc; if (delta <=3D 0) return; + amt =3D sk_mem_pages(delta); sk_forward_alloc_add(sk, amt << PAGE_SHIFT); - sk_memory_allocated_add(sk, amt); =20 - if (mem_cgroup_sk_enabled(sk)) + if (mem_cgroup_sk_enabled(sk)) { mem_cgroup_sk_charge(sk, amt, gfp_memcg_charge() | __GFP_NOFAIL); + + if (mem_cgroup_sk_isolated(sk)) + return; + } + + sk_memory_allocated_add(sk, amt); } =20 /* Send a FIN. The caller locks the socket for us. --=20 2.50.0.727.gbf7dc18ff4-goog