From nobody Tue Dec 16 17:55:57 2025 Received: from mail-pl1-f182.google.com (mail-pl1-f182.google.com [209.85.214.182]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E1AA553BE for ; Sun, 21 Jul 2024 12:52:29 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.182 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1721566351; cv=none; b=jf9ktlHghShNXB9KWRu0b6lda3pnUC9ane1aBVxtOAZ48G1ZjEogEs4jWQfPQAIZNYFNWI62CSabrQcj63rvzxcHlNJgPi7pT8QzgEC4ttgYencSkUaU2UUiD80w96yLRLxMn9xEHgfS+iiQLVY2Pk3hRhyu1qZzSudRnC0L34k= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1721566351; c=relaxed/simple; bh=gzRYSeel4EgQhls0JgRSNs/ESZQm9syKAiLAxunmX2Y=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=WtPocIwJRpitU5nbdN3wPLQR7hBMz5da/facy6ejTRmznITXL7kOYo09CTZSTefiE4T4wFGNNLRCeXUM2jFpFcm8cB57GuM9YzNTHc2O67+HmzAxlV4XKm7THLkkJQr2aHxIEuHJwnXJ5gEFSgzK6dqOQDDLc4dP1mm9oMvPIIk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com; spf=pass smtp.mailfrom=bytedance.com; dkim=pass (2048-bit key) header.d=bytedance.com header.i=@bytedance.com header.b=CP1/Cp87; arc=none smtp.client-ip=209.85.214.182 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=bytedance.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=bytedance.com header.i=@bytedance.com header.b="CP1/Cp87" Received: by mail-pl1-f182.google.com with SMTP id d9443c01a7336-1fc5296e214so27066805ad.0 for ; Sun, 21 Jul 2024 05:52:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1721566349; x=1722171149; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=B22IviPvWu7J2ChPL0whByR8fihGBAS7a0j2bozEE5s=; b=CP1/Cp87zUDuI/pLr7AzsMaggfOV8un3uVwe6yROVsA1tDHYBuvxOTgB+x0dsKQGFN h/RWOy//4zeqADYWkIOnswdG0kUyTN/XmJglF0r3E23KjJnOmlXFFmgxhNbsuytva7li +qJnfCTE6y9jFVoNGSh3Idgnqmtzw6CkE7h/5d3V3I7KH+fyR9o/mfp2417vjsW/BA5R IZFXdoUG0K4JLPiHEe7kKMS1SbeIaVkyXYn8uvILdL2DEMRrePuSTrkqed/weD8Kbg9n OnIh41dq7Y5BzWcAXgYQ4yVU73+1eH9eAFkCoCsCy2+IaNapsm/6UkYCBIeCv9snhIB7 gf7w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1721566349; x=1722171149; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=B22IviPvWu7J2ChPL0whByR8fihGBAS7a0j2bozEE5s=; b=IKicQohEFG2LJHx+Y4eOxEp72/2Nv2MSySRtJezx8+0pK1e2RN6NWwOkjdY8q0y62u zfn0hrOqBXD0M0S8rYfkomRp8duZfVFuUmUYzNOvhV2zBNeMnGrG8xDErEgNBI6BwZrD qwO+SbGDA+4oLRmtnrRM0BxouqerX884wCas/bkiPOVEud+xMb+eL4H7kOIEUh/8no3O 7zol1Ti/3QqyKaywA9PtxEhQnGhepYAakuyG2sT3U/IvIRb2HvKLX7UoMZpJwXlnS5jE RPzQtc/FQy0UWDf5GEl/aoid0eGksZxeaobQaCsmPmWeqpakmkTaUC7qo+ahLE9wgEVG nWxA== X-Forwarded-Encrypted: i=1; AJvYcCViIgUeuykLT1jyINf5X49XjDFyo10cbm8jejhzJZkS8un4Nkbn4GzblQw4cBgqdA93ffKOz3g975G/USig66wAFIo3vFRGQl9Go/1g X-Gm-Message-State: AOJu0Yynil8trXgWt4f5oB3BZNaGbg+FqwxEE7bF/pJAuIdc6jPd+xb5 h471HJfVSf99HFw1R/v+7e//8PwePVIlSRBq0WqPV84cVX8SQacMV7j8OrppLVA= X-Google-Smtp-Source: AGHT+IFM5R0O7O1J8rP+6fMBuhIH4xHgNivDmKp3nKwR0SDc8Av8EYPsELvcNQ1UjIin3wPu5seB3Q== X-Received: by 2002:a17:902:d489:b0:1fd:7c8a:be3e with SMTP id d9443c01a7336-1fd7c8ad873mr37802555ad.36.1721566349126; Sun, 21 Jul 2024 05:52:29 -0700 (PDT) Received: from YGFVJ29LDD.bytedance.net ([139.177.225.235]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-1fd6f472366sm35871525ad.260.2024.07.21.05.52.23 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Sun, 21 Jul 2024 05:52:28 -0700 (PDT) From: Chuyi Zhou To: mingo@redhat.com, peterz@infradead.org, juri.lelli@redhat.com, vincent.guittot@linaro.org, dietmar.eggemann@arm.com, rostedt@goodmis.org, bsegall@google.com, mgorman@suse.de, vschneid@redhat.com Cc: chengming.zhou@linux.dev, linux-kernel@vger.kernel.org, joshdon@google.com, Chuyi Zhou Subject: [PATCH 1/2] sched/fair: Decrease cfs bandwidth usage in task_group destruction Date: Sun, 21 Jul 2024 20:52:07 +0800 Message-Id: <20240721125208.5348-2-zhouchuyi@bytedance.com> X-Mailer: git-send-email 2.37.0 (Apple Git-136) In-Reply-To: <20240721125208.5348-1-zhouchuyi@bytedance.com> References: <20240721125208.5348-1-zhouchuyi@bytedance.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" The static key __cfs_bandwidth_used is used to indicate whether bandwidth control is enabled in the system. Currently, it is only decreased when a task group disables bandwidth control. This is incorrect because if there was a task group in the past that enabled bandwidth control, the __cfs_bandwidth_used will never go to zero, even if there are no task_group using bandwidth control now. This patch tries to fix this issue by decrsasing bandwidth usage in destroy_cfs_bandwidth(). Signed-off-by: Chuyi Zhou --- kernel/sched/fair.c | 3 +++ 1 file changed, 3 insertions(+) diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c index b1e07ce90284..7ad50dc31a93 100644 --- a/kernel/sched/fair.c +++ b/kernel/sched/fair.c @@ -6447,6 +6447,9 @@ static void destroy_cfs_bandwidth(struct cfs_bandwidt= h *cfs_b) hrtimer_cancel(&cfs_b->period_timer); hrtimer_cancel(&cfs_b->slack_timer); =20 + if (cfs_b->quota !=3D RUNTIME_INF) + cfs_bandwidth_usage_dec(); + /* * It is possible that we still have some cfs_rq's pending on a CSD * list, though this race is very rare. In order for this to occur, we --=20 2.20.1 From nobody Tue Dec 16 17:55:57 2025 Received: from mail-oi1-f176.google.com (mail-oi1-f176.google.com [209.85.167.176]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 16A9153BE for ; Sun, 21 Jul 2024 12:53:04 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.167.176 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1721566386; cv=none; b=hTC7lYEkTta6UP0apub/h866vn1UfF89PtscELaW0rx2thHebZaxZMwNeElL59b0BYzhg9NSDXcp6fnHN9kvwSeVrKxHt0HH/wBVTjGb9M9IyieuaNJeJoklxFzctiSSy8SM0YJBMm8g42ReWMUbIztfZWgLMV82No0VKKDzuqU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1721566386; c=relaxed/simple; bh=rd9yyAOuaPjKVTKfd1c64lmlFO84QaWok+uMlwK7TOc=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=PTj8Ywg4+hUoP22aDNh8vg3nAM533+Jy7PaGAE1UlzRjVw5a3jFmOUaWXCLhscAimNpTybeGa1ADlVUMdINjEV6vvabZoXB0MyZSS6RykKphTGajicObV8OfKaGqpUVhMii03kZxoV5LnQNMkO1rl01AtIxVYcuJo8rJOv38m7w= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com; spf=pass smtp.mailfrom=bytedance.com; dkim=pass (2048-bit key) header.d=bytedance.com header.i=@bytedance.com header.b=CYk7ll/v; arc=none smtp.client-ip=209.85.167.176 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=bytedance.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=bytedance.com header.i=@bytedance.com header.b="CYk7ll/v" Received: by mail-oi1-f176.google.com with SMTP id 5614622812f47-3d9bcb47182so2060932b6e.3 for ; Sun, 21 Jul 2024 05:53:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1721566384; x=1722171184; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=i90tat1on8IktACrStM2Flu+kAGDHgBdM6VXZCQtR7U=; b=CYk7ll/vQr8IR+F+K4ffAVQzVFQPB8X14qgI3Gw9KWSwi9NL2meG8gEJRKhwvsEUGx 69Sb8+SqdFmOAFebvQSTm1F/HEfK3ez0YFmzdtVBB8980HqjQfc1jdXd2tvc8N3NVxhR hxPpMAgrwHxIqRNOXtvE88lC2yyIEIi3vlzHozs/me433akIV7A+bZ8cty/yy/owdsek rO1WvQu4QFR4P9M8DDVrTCOF09BnozKeIe354XFHps/5e/S9+j0o8D9ZVM7ZC7VYoJdO oAaMK9vU+UgR7CF00aerBtBXSxZAycb0d/wsLtGHieqAB8XrDBpoeyg7LK0+1qgBTS9r BDvg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1721566384; x=1722171184; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=i90tat1on8IktACrStM2Flu+kAGDHgBdM6VXZCQtR7U=; b=RFiDkKhEs6LxJQ4wcvhS9+6RBVmA5LC0aXR2Ckm9mdX5XHGrPnJuL6m5pKvlUdIcfW 6mDFhzCkGBkdKXMeFpoGmLiLkKsr6kAxTXJGO+dGrYkIK0OAqAQYGL4TyDZXzEKYow9J Pqv/4eI1p40fiAiDYUPCOI/rjFEnTKVkYWR9rMR7ozoc25X1kgxE/9y3P5wLn+40luHr BUXJbT2KCPImXDOEfF/HWzz7JST7yEYQLAoDXzXD6f6b1zcSPRFwEj2BGAhoInuRani6 JESN2y87ae8iL1gVxni+9+7cvWQvy3D0iFlIe6zs6V6k31ghu4nsaJz0SNw08YrSuNay 2Kkw== X-Forwarded-Encrypted: i=1; AJvYcCWgjPWpoOjLvDowhK/6+qZiNUJVy9msy/PIt6CeE0bAh6ccAlLGoE7vV/oZPB9gu0zUCTkeGDEh95ecAs1uwpGO99/SisgWbml59DRY X-Gm-Message-State: AOJu0Yy4fhjLjze16La1kfwS3aIhFWFa+K84Ndek9gURL94peInNyg7+ AiLlUj4W39xRf3c4orPd6yIKdVVRdRz9kDcqCxOoBvbCGQ5lF67j7Fhq1HstBkY= X-Google-Smtp-Source: AGHT+IHHPPB5MeTECgKUm/IaTWBdefeqtodW7G8DnMTUGOtt21y6JS6+uY7jp9S0ihTlBFRw1c+OqA== X-Received: by 2002:a05:6808:14c7:b0:3da:c428:1c51 with SMTP id 5614622812f47-3dae9723bc7mr4142862b6e.8.1721566354402; Sun, 21 Jul 2024 05:52:34 -0700 (PDT) Received: from YGFVJ29LDD.bytedance.net ([139.177.225.235]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-1fd6f472366sm35871525ad.260.2024.07.21.05.52.29 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Sun, 21 Jul 2024 05:52:34 -0700 (PDT) From: Chuyi Zhou To: mingo@redhat.com, peterz@infradead.org, juri.lelli@redhat.com, vincent.guittot@linaro.org, dietmar.eggemann@arm.com, rostedt@goodmis.org, bsegall@google.com, mgorman@suse.de, vschneid@redhat.com Cc: chengming.zhou@linux.dev, linux-kernel@vger.kernel.org, joshdon@google.com, Chuyi Zhou Subject: [PATCH 2/2] sched/core: Avoid unnecessary update in tg_set_cfs_bandwidth Date: Sun, 21 Jul 2024 20:52:08 +0800 Message-Id: <20240721125208.5348-3-zhouchuyi@bytedance.com> X-Mailer: git-send-email 2.37.0 (Apple Git-136) In-Reply-To: <20240721125208.5348-1-zhouchuyi@bytedance.com> References: <20240721125208.5348-1-zhouchuyi@bytedance.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" In the kubernetes production environment, we have observed a high frequency of writes to cpu.max, approximately every 2~4 seconds for each cgroup, with the same value being written each time. This can result in unnecessary overhead, especially on machines with a large number of CPUs and cgroups. This is because kubelet and runc attempt to persist resource configurations through frequent updates with same value in this manner. While optimizations can be made to kubelet and runc to avoid such overhead(e.g. check the current value of cpu request/limit before writing to cpu.max), it is still worth to bail out from tg_set_cfs_bandwidth() if we attempt to update with the same value. Signed-off-by: Chuyi Zhou --- kernel/sched/core.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/kernel/sched/core.c b/kernel/sched/core.c index 6d35c48239be..4db3ef2a703b 100644 --- a/kernel/sched/core.c +++ b/kernel/sched/core.c @@ -9081,6 +9081,8 @@ static int tg_set_cfs_bandwidth(struct task_group *tg= , u64 period, u64 quota, burst + quota > max_cfs_runtime)) return -EINVAL; =20 + if (cfs_b->period =3D=3D ns_to_ktime(period) && cfs_b->quota =3D=3D quota= && cfs_b->burst =3D=3D burst) + return 0; /* * Prevent race between setting of cfs_rq->runtime_enabled and * unthrottle_offline_cfs_rqs(). --=20 2.20.1