From nobody Fri Feb 13 15:42:19 2026 Received: from mail-pg1-f182.google.com (mail-pg1-f182.google.com [209.85.215.182]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 172D36BFA7 for ; Thu, 18 Apr 2024 06:26:13 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.215.182 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1713421576; cv=none; b=nLlvO4DPjOpeaZsBkBg99DbkOsdyNeRP0cNKxxkoMhJc7GkoFLRCJkoqOrhVodsOAxpisjt0nB9wUhvVjd/S/ASfHA4bDHmqj2MWFIc4LzrZVF4tU8g0A77sXz6/AMS4X7tInV2z2kXoC4gVr9eJvfl5cZ6VI9gFV0jPsu14ACs= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1713421576; c=relaxed/simple; bh=ohLO789sHfFgMKSOwGDlzSy99Mj1iOqRNgRJnS8QIiA=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=LRj3oRExa2/6+aGvub5WoMDpjG7WO1tb86sYzWI7zib+rozh/ZeRORva5bJYNPnaGIKvZX/ouAixWQxInlVl/Q++bkPw4CNSf0HIqbx2BUZbn2zwd/DJjegCY80ggz/uKoTr3OJzxwVxa7pg12BDunQSA/LuhNTXuqeWO2q2kDo= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com; spf=pass smtp.mailfrom=bytedance.com; dkim=pass (2048-bit key) header.d=bytedance.com header.i=@bytedance.com header.b=iVHKHLCY; arc=none smtp.client-ip=209.85.215.182 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=bytedance.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=bytedance.com header.i=@bytedance.com header.b="iVHKHLCY" Received: by mail-pg1-f182.google.com with SMTP id 41be03b00d2f7-517ab9a4a13so341274a12.1 for ; Wed, 17 Apr 2024 23:26:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1713421573; x=1714026373; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=wbU6FsIuncluFPwH7VPsX/cWOPAMxIJmhU49fecmUC4=; b=iVHKHLCYAvsLjb3uq33ZkepPxXPC5fT3FM8HR/8c0UsUNAEy5A38yVI29GcvI6CUB7 n033q8L9UM0McygOxv0pi5WCHl0o+VCpUuqeBVZsIbmSN57JMc6do5kjLbQ5jL1ylsdL 6KqfBD9RniKoU+NnURrLW3agiM40XVWRpw6C7MFgZ3PpAqmnlLlPYaquwIdlSRA52aC5 QUlGqaUQCqIScdiDIxKBef2ph8Psgaa0NnzHxGHfO8/sMZdvONQ8D5mSN9TdSGrR3CJU 06X+Su3KHqV73cO4FUDUklbPk7SC5mq/R40USFNgvxcHdk+DohK6P1Y7VOv7EuWSSg2I 5dDA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1713421573; x=1714026373; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=wbU6FsIuncluFPwH7VPsX/cWOPAMxIJmhU49fecmUC4=; b=o4O7rpC+E+cOCBl7zPevtj9RuouuxNsFMC6kyCfQFfqyzSuwoz9b0+80Pqekc8hVLw qlUfREitbjA7zRVAUGRqVp5ziKQBB6oHnGDxgqZ1xGLiPgfWUKhpiN9buOPIZDHsb3uE mc9dAFclO/qFxbqLHeLPyGMPfzIRJxJeYXFmitUH5GQsZSIZ8hIbCNPvgxmKGnxs0864 X7H11NXFVsO++7zJAQfUjRYmQzzP9mzfvU4z2V51QU/MxGvGJHRvgDyOBui1V93ZztpP EeX52PqFTPRrAKFWo62yXhMNorjYvRBzAR8fzSMImHqctrdvjEv4eslGUNgTS1OkN6qz 7OWA== X-Gm-Message-State: AOJu0YxrpWUIqAN1AqKY9QRwNW8HbDvpm3sHLQHzrGlyzPH8/1jdOLmU rTGSNVwTj04xSIT8feH23+sesbGUsmh6aY8q1PtvyK3B24o0y6eLFSUAyetqvmcFkF7javXlWUV 1 X-Google-Smtp-Source: AGHT+IG8KbPnW3d1OwSGwOGuRvAB4kzTNDzAERiB9f7OxYdoLKYRlUD9DImfcrUf+Wr56E8/G1LSNw== X-Received: by 2002:a05:6a21:2709:b0:1a3:5c61:5ec2 with SMTP id rm9-20020a056a21270900b001a35c615ec2mr2398878pzb.16.1713421573195; Wed, 17 Apr 2024 23:26:13 -0700 (PDT) Received: from libai.bytedance.net ([61.213.176.11]) by smtp.gmail.com with ESMTPSA id s21-20020a632155000000b005cd8044c6fesm666392pgm.23.2024.04.17.23.26.10 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 17 Apr 2024 23:26:12 -0700 (PDT) From: zhenwei pi To: linux-kernel@vger.kernel.org, linux-mm@kvack.org, virtualization@lists.linux.dev Cc: mst@redhat.com, david@redhat.com, jasowang@redhat.com, xuanzhuo@linux.alibaba.com, akpm@linux-foundation.org, zhenwei pi Subject: [PATCH 1/3] virtio_balloon: introduce oom-kill invocations Date: Thu, 18 Apr 2024 14:26:00 +0800 Message-Id: <20240418062602.1291391-2-pizhenwei@bytedance.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240418062602.1291391-1-pizhenwei@bytedance.com> References: <20240418062602.1291391-1-pizhenwei@bytedance.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" When the guest OS runs under critical memory pressure, the guest starts to kill processes. A guest monitor agent may scan 'oom_kill' from /proc/vmstat, and reports the OOM KILL event. However, the agent may be killed and we will loss this critical event(and the later events). For now we can also grep for magic words in guest kernel log from host side. Rather than this unstable way, virtio balloon reports OOM-KILL invocations instead. Signed-off-by: zhenwei pi Acked-by: David Hildenbrand --- drivers/virtio/virtio_balloon.c | 1 + include/uapi/linux/virtio_balloon.h | 6 ++++-- 2 files changed, 5 insertions(+), 2 deletions(-) diff --git a/drivers/virtio/virtio_balloon.c b/drivers/virtio/virtio_balloo= n.c index 1f5b3dd31fcf..fd19934a847f 100644 --- a/drivers/virtio/virtio_balloon.c +++ b/drivers/virtio/virtio_balloon.c @@ -337,6 +337,7 @@ static unsigned int update_balloon_stats(struct virtio_= balloon *vb) pages_to_bytes(events[PSWPOUT])); update_stat(vb, idx++, VIRTIO_BALLOON_S_MAJFLT, events[PGMAJFAULT]); update_stat(vb, idx++, VIRTIO_BALLOON_S_MINFLT, events[PGFAULT]); + update_stat(vb, idx++, VIRTIO_BALLOON_S_OOM_KILL, events[OOM_KILL]); #ifdef CONFIG_HUGETLB_PAGE update_stat(vb, idx++, VIRTIO_BALLOON_S_HTLB_PGALLOC, events[HTLB_BUDDY_PGALLOC]); diff --git a/include/uapi/linux/virtio_balloon.h b/include/uapi/linux/virti= o_balloon.h index ddaa45e723c4..b17bbe033697 100644 --- a/include/uapi/linux/virtio_balloon.h +++ b/include/uapi/linux/virtio_balloon.h @@ -71,7 +71,8 @@ struct virtio_balloon_config { #define VIRTIO_BALLOON_S_CACHES 7 /* Disk caches */ #define VIRTIO_BALLOON_S_HTLB_PGALLOC 8 /* Hugetlb page allocations */ #define VIRTIO_BALLOON_S_HTLB_PGFAIL 9 /* Hugetlb page allocation failu= res */ -#define VIRTIO_BALLOON_S_NR 10 +#define VIRTIO_BALLOON_S_OOM_KILL 10 /* OOM killer invocations */ +#define VIRTIO_BALLOON_S_NR 11 =20 #define VIRTIO_BALLOON_S_NAMES_WITH_PREFIX(VIRTIO_BALLOON_S_NAMES_prefix) = { \ VIRTIO_BALLOON_S_NAMES_prefix "swap-in", \ @@ -83,7 +84,8 @@ struct virtio_balloon_config { VIRTIO_BALLOON_S_NAMES_prefix "available-memory", \ VIRTIO_BALLOON_S_NAMES_prefix "disk-caches", \ VIRTIO_BALLOON_S_NAMES_prefix "hugetlb-allocations", \ - VIRTIO_BALLOON_S_NAMES_prefix "hugetlb-failures" \ + VIRTIO_BALLOON_S_NAMES_prefix "hugetlb-failures", \ + VIRTIO_BALLOON_S_NAMES_prefix "oom-kills" \ } =20 #define VIRTIO_BALLOON_S_NAMES VIRTIO_BALLOON_S_NAMES_WITH_PREFIX("") --=20 2.34.1 From nobody Fri Feb 13 15:42:19 2026 Received: from mail-pf1-f172.google.com (mail-pf1-f172.google.com [209.85.210.172]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9309875801 for ; Thu, 18 Apr 2024 06:26:17 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.172 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1713421578; cv=none; b=RE24aHMZCAYA2kkSzbc/z58jFDFeUq60HoUn5Ry1rFfqEYyEjH+xW+ESQmVQpmF0apAmaaQcUMQrOnxUImt+pxqfnQZwlGES7M/kotLJsYKNb2zc+4Pj4SEK3YExF+mFZyRD8B1Ol+M+/9oBiK5ZSt/1JGGyK6qlNktFoVkW7lE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1713421578; c=relaxed/simple; bh=Xzbgih7B8aIf9LhXB1XHaCkjr0jHp8G/PviFgcyTSdE=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=bBL73AeDOyH03FprMnXXePwCWxj7F9WVv7AKb1Xsl3cLsbZdjcL4hKFwXtdPm6VGqHQrXY4fkRpCIaA3xS2kcqy+SVlsSi2QGf+9RqYwX3//0rF+3O0/Aw6xAYMq1TKSkuLtZDl1hZ48y3BUYpYHALvRLoupAVqe3pwLtd/Mid0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com; spf=pass smtp.mailfrom=bytedance.com; dkim=pass (2048-bit key) header.d=bytedance.com header.i=@bytedance.com header.b=FvohFNb8; arc=none smtp.client-ip=209.85.210.172 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=bytedance.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=bytedance.com header.i=@bytedance.com header.b="FvohFNb8" Received: by mail-pf1-f172.google.com with SMTP id d2e1a72fcca58-6ecf1bb7f38so566242b3a.0 for ; Wed, 17 Apr 2024 23:26:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1713421576; x=1714026376; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=qLD61XepBARAsgwAEEryGhLKo22WgyhNR7D0Io1kaZU=; b=FvohFNb83f7um1yMuROvMcDV88ouDia5Lne4LUeEQ+AmptfrQ1G23pOFJiFWJ0igcw mxam4BjJin+ruGZ8L25N/X4tqMPLMS5vFaaP6OcwghpbwM04R9WiWlTifbDTGhDa42fw hQ3IXyVER6MQJiv2tOu/MGH0b2Xi6HhRJi/8u5ZHoLXTGvuYUsmnRD44JlcUJJbOfKv9 uAafSp76FowJ577Vfrg7sYK2tLEjVHylHbQxs2QaKoEPZRfETId5noErNTS9s/+MpjWY zQV6gDziAtOl8P+A7gttFYWrjC2N5YxWnbbdwZj7qF9jmuxocSuT4nHPKHAfcuSpKN/U KVhA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1713421576; x=1714026376; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=qLD61XepBARAsgwAEEryGhLKo22WgyhNR7D0Io1kaZU=; b=nurP1qk4y8FigIDstONJRlsMQtCriH4ENidDaKyjY+U/CduqskBELTzJ7KpXW+m7Fn PixhAfDUYfj92KBdV/vOBNExfPgnbNKJPSlUIX/c3SuvRp9MvAETnFh1Low2r1HBEI1i SHqnDD/xQKiAs2omMQ2UUTC37aCRTn2u2p3TXir11PKKnj6v6YTPE/dSCgJWNWhHjLD6 gJYPb7Me56i9PSvMWJxZCAJd8JfqEf+C96pvveKwqoFI+muPiqvHMYFO0w3JWOCFHtkv f9Tr2WtcDdjkHdz9dhyx2/OzlQlMT+cLcCd0uIhtzRXhbowrpFJ2SHveuxSrBcbWkMZX 68lg== X-Gm-Message-State: AOJu0YyBY8QR5ngbI8ee80UWNuZmZhPWWgiHaVYQzGotabs3iByD4Ukv 7WjQfjsKemRZBG9hRVbqHlCbvhOAvjuijbr5cfmpgEUPf3+iPVUG202ZcxceFDqSDq1nilXmSxU G X-Google-Smtp-Source: AGHT+IExq7y1kqyqGXewPq6tqKPgeINzrgW8X9DzhXoz5Wq5smACLbcQzHv5/LkcKr/osJEn2qOGzg== X-Received: by 2002:a05:6a20:9717:b0:1a9:dcd9:9756 with SMTP id hr23-20020a056a20971700b001a9dcd99756mr2120694pzc.16.1713421576513; Wed, 17 Apr 2024 23:26:16 -0700 (PDT) Received: from libai.bytedance.net ([61.213.176.11]) by smtp.gmail.com with ESMTPSA id s21-20020a632155000000b005cd8044c6fesm666392pgm.23.2024.04.17.23.26.13 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 17 Apr 2024 23:26:16 -0700 (PDT) From: zhenwei pi To: linux-kernel@vger.kernel.org, linux-mm@kvack.org, virtualization@lists.linux.dev Cc: mst@redhat.com, david@redhat.com, jasowang@redhat.com, xuanzhuo@linux.alibaba.com, akpm@linux-foundation.org, zhenwei pi Subject: [PATCH 2/3] virtio_balloon: introduce memory allocation stall counter Date: Thu, 18 Apr 2024 14:26:01 +0800 Message-Id: <20240418062602.1291391-3-pizhenwei@bytedance.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240418062602.1291391-1-pizhenwei@bytedance.com> References: <20240418062602.1291391-1-pizhenwei@bytedance.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Memory allocation stall counter represents the performance/latency of memory allocation, expose this counter to the host side by virtio balloon device via out-of-bound way. Signed-off-by: zhenwei pi --- drivers/virtio/virtio_balloon.c | 20 +++++++++++++++++++- include/uapi/linux/virtio_balloon.h | 6 ++++-- 2 files changed, 23 insertions(+), 3 deletions(-) diff --git a/drivers/virtio/virtio_balloon.c b/drivers/virtio/virtio_balloo= n.c index fd19934a847f..e88e6573afa5 100644 --- a/drivers/virtio/virtio_balloon.c +++ b/drivers/virtio/virtio_balloon.c @@ -321,7 +321,7 @@ static unsigned int update_balloon_stats(struct virtio_= balloon *vb) unsigned long events[NR_VM_EVENT_ITEMS]; struct sysinfo i; unsigned int idx =3D 0; - long available; + long available, stall =3D 0; unsigned long caches; =20 all_vm_events(events); @@ -338,6 +338,24 @@ static unsigned int update_balloon_stats(struct virtio= _balloon *vb) update_stat(vb, idx++, VIRTIO_BALLOON_S_MAJFLT, events[PGMAJFAULT]); update_stat(vb, idx++, VIRTIO_BALLOON_S_MINFLT, events[PGFAULT]); update_stat(vb, idx++, VIRTIO_BALLOON_S_OOM_KILL, events[OOM_KILL]); + + /* sum all the stall events */ +#ifdef CONFIG_ZONE_DMA + stall +=3D events[ALLOCSTALL_DMA]; +#endif +#ifdef CONFIG_ZONE_DMA32 + stall +=3D events[ALLOCSTALL_DMA32]; +#endif +#ifdef CONFIG_HIGHMEM + stall +=3D events[ALLOCSTALL_HIGH]; +#endif +#ifdef CONFIG_ZONE_DEVICE + stall +=3D events[ALLOCSTALL_DEVICE]; +#endif + stall +=3D events[ALLOCSTALL_NORMAL]; + stall +=3D events[ALLOCSTALL_MOVABLE]; + update_stat(vb, idx++, VIRTIO_BALLOON_S_ALLOC_STALL, stall); + #ifdef CONFIG_HUGETLB_PAGE update_stat(vb, idx++, VIRTIO_BALLOON_S_HTLB_PGALLOC, events[HTLB_BUDDY_PGALLOC]); diff --git a/include/uapi/linux/virtio_balloon.h b/include/uapi/linux/virti= o_balloon.h index b17bbe033697..487b893a160e 100644 --- a/include/uapi/linux/virtio_balloon.h +++ b/include/uapi/linux/virtio_balloon.h @@ -72,7 +72,8 @@ struct virtio_balloon_config { #define VIRTIO_BALLOON_S_HTLB_PGALLOC 8 /* Hugetlb page allocations */ #define VIRTIO_BALLOON_S_HTLB_PGFAIL 9 /* Hugetlb page allocation failu= res */ #define VIRTIO_BALLOON_S_OOM_KILL 10 /* OOM killer invocations */ -#define VIRTIO_BALLOON_S_NR 11 +#define VIRTIO_BALLOON_S_ALLOC_STALL 11 /* Stall count of memory allocat= oin */ +#define VIRTIO_BALLOON_S_NR 12 =20 #define VIRTIO_BALLOON_S_NAMES_WITH_PREFIX(VIRTIO_BALLOON_S_NAMES_prefix) = { \ VIRTIO_BALLOON_S_NAMES_prefix "swap-in", \ @@ -85,7 +86,8 @@ struct virtio_balloon_config { VIRTIO_BALLOON_S_NAMES_prefix "disk-caches", \ VIRTIO_BALLOON_S_NAMES_prefix "hugetlb-allocations", \ VIRTIO_BALLOON_S_NAMES_prefix "hugetlb-failures", \ - VIRTIO_BALLOON_S_NAMES_prefix "oom-kills" \ + VIRTIO_BALLOON_S_NAMES_prefix "oom-kills", \ + VIRTIO_BALLOON_S_NAMES_prefix "alloc-stalls" \ } =20 #define VIRTIO_BALLOON_S_NAMES VIRTIO_BALLOON_S_NAMES_WITH_PREFIX("") --=20 2.34.1 From nobody Fri Feb 13 15:42:19 2026 Received: from mail-ot1-f45.google.com (mail-ot1-f45.google.com [209.85.210.45]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D73A179950 for ; Thu, 18 Apr 2024 06:26:20 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.45 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1713421582; cv=none; b=tG4x40t26s50c4LyOkyouJqDmBCi5P/M/vh3pbMycSntLB7Lczu/kuYmnIb24ydMCJ+7gAVADiRtkB5AcifdYFT6BomVjT1Gw0LMxc8cuKXA+5ER0+yghC4wxHLqnrQ+QVSALq2mo3KxkB2kUADrKto0D55UG/xFYtt3L2GK0us= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1713421582; c=relaxed/simple; bh=iQYgmz5Gef1EaE1UJeXMcgsfVqRfmIwziXDAGoA5Vos=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=MtWdwNPfomOHuFRUHgTmVkJWnSdXC+QQtQbLA4EnQ1JuKzUDVa8xUykd3/it4lsnV7yoHkgboQLzMo3N4qJgqpNPKKj5DcLpUbuPJ1Wc4rkg7W517777EWMhQQTZ4llUxgyikRxEyeSJ7da9E+WijCvApj81MHr+dnMrF1t7igI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com; spf=pass smtp.mailfrom=bytedance.com; dkim=pass (2048-bit key) header.d=bytedance.com header.i=@bytedance.com header.b=HOgAFCV7; arc=none smtp.client-ip=209.85.210.45 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=bytedance.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=bytedance.com header.i=@bytedance.com header.b="HOgAFCV7" Received: by mail-ot1-f45.google.com with SMTP id 46e09a7af769-6eb7ee5a776so267976a34.3 for ; Wed, 17 Apr 2024 23:26:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1713421580; x=1714026380; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=x0vNJuhAwUKc+UYf59Hf5S0kNxOXWBcwSUDB4ktR3Zw=; b=HOgAFCV7xMS2sBWA27NyLnfgnQXnoSqCkjeFKJjfVa+5MVvyi9ePkKJulIgEG4CB0h g/wsHYnqTBiXk9qtVTXfsg9GLVp+J8KPxT/HJm1oDrfx8v82rhwvb3m+OmOQ+mEIGTA5 9GMvQLjFe6boofkLv0tTTr8PCNGo17NGbclWgET3b4u9qRpGTnDN1VLtg4cPfwjxwHXl WJ5495SFkcrF4FBIQgWx9WMSWDy/ufZNtcbMfF+qCOdHR1R286pprM2bj/bJz9H38arP XF+hCgXqnLzgTBiTZxxTASr9Dl+eaztLXB5gc6Eu07VI4z2tbrd7524Lpcyp7NYWvi5X PfXw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1713421580; x=1714026380; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=x0vNJuhAwUKc+UYf59Hf5S0kNxOXWBcwSUDB4ktR3Zw=; b=p9ZvGszBvW6+mQu6e0n8YjV/VhXxMKXTIgk0xVLyf092yNQSZApP9YigNDvdPiEOAV tnN5Enhqt+2xVZ577vT7JQgOjk5j4qfLGWPROrMBBx5XELp3DyJXTL/62jkTKB+kQPKZ EdNPwfEep3ovxheGBVukfb0kQalMbG4LKguLtHFVrk6bBR6JHab+Kk9jVh8GgsPN89J3 RJsNWRa6XiZwYpvCwwQ0GQVM2dOyZwqNxV/8aFZkwo5dTygjOQL/H7Jh20EEv9S7XAEu Tb7jrK/TcK6dm3pD53e3MWs7Z0SYVg3sXdk23K6fFLfLbsIfTo/BbQ2vzxsuMV9N8P5d +V+Q== X-Gm-Message-State: AOJu0YwsOxOEScCA1dGe0euk2jGLgPad7LztA9mEbJfHyp9dbqm+ouyz BHg0zpcIT8XiUXN/3ZYN1wA+jP1Xlw6HB9tcJCC5dnoLPkvkECuqEnwv4xrWzTVJm8f0MCVFuSm f X-Google-Smtp-Source: AGHT+IHHR6Htepai94lycGDse6wf1IdRutz/3r1ea6ZF/gP/tH+hAUh/tHbzPsYdqQF4kLBlWMl4TA== X-Received: by 2002:a9d:6a90:0:b0:6eb:bbc6:9af0 with SMTP id l16-20020a9d6a90000000b006ebbbc69af0mr2171391otq.12.1713421579905; Wed, 17 Apr 2024 23:26:19 -0700 (PDT) Received: from libai.bytedance.net ([61.213.176.11]) by smtp.gmail.com with ESMTPSA id s21-20020a632155000000b005cd8044c6fesm666392pgm.23.2024.04.17.23.26.16 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 17 Apr 2024 23:26:19 -0700 (PDT) From: zhenwei pi To: linux-kernel@vger.kernel.org, linux-mm@kvack.org, virtualization@lists.linux.dev Cc: mst@redhat.com, david@redhat.com, jasowang@redhat.com, xuanzhuo@linux.alibaba.com, akpm@linux-foundation.org, zhenwei pi Subject: [PATCH 3/3] virtio_balloon: introduce memory scan/reclaim info Date: Thu, 18 Apr 2024 14:26:02 +0800 Message-Id: <20240418062602.1291391-4-pizhenwei@bytedance.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240418062602.1291391-1-pizhenwei@bytedance.com> References: <20240418062602.1291391-1-pizhenwei@bytedance.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Expose memory scan/reclaim information to the host side via virtio balloon device. Now we have a metric to analyze the memory performance: y: counter increases n: counter does not changes h: the rate of counter change is high l: the rate of counter change is low OOM: VIRTIO_BALLOON_S_OOM_KILL STALL: VIRTIO_BALLOON_S_ALLOC_STALL ASCAN: VIRTIO_BALLOON_S_SCAN_ASYNC DSCAN: VIRTIO_BALLOON_S_SCAN_DIRECT ARCLM: VIRTIO_BALLOON_S_RECLAIM_ASYNC DRCLM: VIRTIO_BALLOON_S_RECLAIM_DIRECT - OOM[y], STALL[*], ASCAN[*], DSCAN[*], ARCLM[*], DRCLM[*]: the guest runs under really critial memory pressure - OOM[n], STALL[h], ASCAN[*], DSCAN[l], ARCLM[*], DRCLM[l]: the memory allocation stalls due to cgroup, not the global memory pressure. - OOM[n], STALL[h], ASCAN[*], DSCAN[h], ARCLM[*], DRCLM[h]: the memory allocation stalls due to global memory pressure. The performance gets hurt a lot. A high ratio between DRCLM/DSCAN shows quite effective memory reclaiming. - OOM[n], STALL[h], ASCAN[*], DSCAN[h], ARCLM[*], DRCLM[l]: the memory allocation stalls due to global memory pressure. the ratio between DRCLM/DSCAN gets low, the guest OS is thrashing heavily, the serious case leads poor performance and difficult trouble shooting. Ex, sshd may block on memory allocation when accepting new connections, a user can't login a VM by ssh command. - OOM[n], STALL[n], ASCAN[h], DSCAN[n], ARCLM[l], DRCLM[n]: the low ratio between ARCLM/ASCAN shows that the guest tries to reclaim more memory, but it can't. Once more memory is required in future, it will struggle to reclaim memory. Signed-off-by: zhenwei pi Acked-by: David Hildenbrand --- drivers/virtio/virtio_balloon.c | 9 +++++++++ include/uapi/linux/virtio_balloon.h | 12 ++++++++++-- 2 files changed, 19 insertions(+), 2 deletions(-) diff --git a/drivers/virtio/virtio_balloon.c b/drivers/virtio/virtio_balloo= n.c index e88e6573afa5..bc9332c1ae85 100644 --- a/drivers/virtio/virtio_balloon.c +++ b/drivers/virtio/virtio_balloon.c @@ -356,6 +356,15 @@ static unsigned int update_balloon_stats(struct virtio= _balloon *vb) stall +=3D events[ALLOCSTALL_MOVABLE]; update_stat(vb, idx++, VIRTIO_BALLOON_S_ALLOC_STALL, stall); =20 + update_stat(vb, idx++, VIRTIO_BALLOON_S_ASYNC_SCAN, + pages_to_bytes(events[PGSCAN_KSWAPD])); + update_stat(vb, idx++, VIRTIO_BALLOON_S_DIRECT_SCAN, + pages_to_bytes(events[PGSCAN_DIRECT])); + update_stat(vb, idx++, VIRTIO_BALLOON_S_ASYNC_RECLAIM, + pages_to_bytes(events[PGSTEAL_KSWAPD])); + update_stat(vb, idx++, VIRTIO_BALLOON_S_DIRECT_RECLAIM, + pages_to_bytes(events[PGSTEAL_DIRECT])); + #ifdef CONFIG_HUGETLB_PAGE update_stat(vb, idx++, VIRTIO_BALLOON_S_HTLB_PGALLOC, events[HTLB_BUDDY_PGALLOC]); diff --git a/include/uapi/linux/virtio_balloon.h b/include/uapi/linux/virti= o_balloon.h index 487b893a160e..ee35a372805d 100644 --- a/include/uapi/linux/virtio_balloon.h +++ b/include/uapi/linux/virtio_balloon.h @@ -73,7 +73,11 @@ struct virtio_balloon_config { #define VIRTIO_BALLOON_S_HTLB_PGFAIL 9 /* Hugetlb page allocation failu= res */ #define VIRTIO_BALLOON_S_OOM_KILL 10 /* OOM killer invocations */ #define VIRTIO_BALLOON_S_ALLOC_STALL 11 /* Stall count of memory allocat= oin */ -#define VIRTIO_BALLOON_S_NR 12 +#define VIRTIO_BALLOON_S_ASYNC_SCAN 12 /* Amount of memory scanned asyn= chronously */ +#define VIRTIO_BALLOON_S_DIRECT_SCAN 13 /* Amount of memory scanned dire= ctly */ +#define VIRTIO_BALLOON_S_ASYNC_RECLAIM 14 /* Amount of memory reclaimed as= ynchronously */ +#define VIRTIO_BALLOON_S_DIRECT_RECLAIM 15 /* Amount of memory reclaimed d= irectly */ +#define VIRTIO_BALLOON_S_NR 16 =20 #define VIRTIO_BALLOON_S_NAMES_WITH_PREFIX(VIRTIO_BALLOON_S_NAMES_prefix) = { \ VIRTIO_BALLOON_S_NAMES_prefix "swap-in", \ @@ -87,7 +91,11 @@ struct virtio_balloon_config { VIRTIO_BALLOON_S_NAMES_prefix "hugetlb-allocations", \ VIRTIO_BALLOON_S_NAMES_prefix "hugetlb-failures", \ VIRTIO_BALLOON_S_NAMES_prefix "oom-kills", \ - VIRTIO_BALLOON_S_NAMES_prefix "alloc-stalls" \ + VIRTIO_BALLOON_S_NAMES_prefix "alloc-stalls", \ + VIRTIO_BALLOON_S_NAMES_prefix "async-scans", \ + VIRTIO_BALLOON_S_NAMES_prefix "direct-scans", \ + VIRTIO_BALLOON_S_NAMES_prefix "async-reclaims", \ + VIRTIO_BALLOON_S_NAMES_prefix "direct-reclaims" \ } =20 #define VIRTIO_BALLOON_S_NAMES VIRTIO_BALLOON_S_NAMES_WITH_PREFIX("") --=20 2.34.1