From nobody Mon Feb 9 17:58:11 2026 Received: from mail-ot1-f44.google.com (mail-ot1-f44.google.com [209.85.210.44]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 619F81CA82 for ; Tue, 23 Apr 2024 03:41:21 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.44 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1713843682; cv=none; b=rCDsvZ20i40GNLveR42fK+Ucvsvcb4Hb7JLp3oh/wZ2bd5Oy2exLjI1BeQ91/d9hCkM4DiPjpeUsMlXoy5+hmBuCr7R1A3415QelM9XEyQiWIAsbryNiin3/hJeuVOxGyiOqdKuhBUKTyfYFTVyGR7UADu/9oZzOIRAWeQUlGVE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1713843682; c=relaxed/simple; bh=iSysrDVuYXg0UMc79wYI1kULCvYs69ZhGh9/V1D5h+s=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=AUz+4ZReggx4D+NfOGc8mhlI5oFQtZKbGpc0fhEZ5yFC5sIMf9c/zeCRGzHr9JRSCsrgcRKIKnKo6Wk4DLULRWmdke4zTkPsi4YgxSUXBWvXuZNaT+zL3Do9DZ7E5QR+KRPtlFp35vQICuy3eXfkAXRr3Xl3zKhoPir0kj6xh8U= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com; spf=pass smtp.mailfrom=bytedance.com; dkim=pass (2048-bit key) header.d=bytedance.com header.i=@bytedance.com header.b=eNp82nlB; arc=none smtp.client-ip=209.85.210.44 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=bytedance.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=bytedance.com header.i=@bytedance.com header.b="eNp82nlB" Received: by mail-ot1-f44.google.com with SMTP id 46e09a7af769-6ea26393116so3362132a34.0 for ; Mon, 22 Apr 2024 20:41:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1713843680; x=1714448480; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=KkBGMpzb7Zyfmx1ZstJgk5Op8ktl/5dFs5p8PlYlyX8=; b=eNp82nlBJNHzj3Yh79uzhq7zJoIKFlmg9pMmQWYf8K/RsEBOyR4ekwf0wM5i4sbX5d Z8nn9f7Uyay3jD5e2r5o3q61beKYFK7i95QtRx+9thKnACyVYU2k2XPSquyR9WdygDJR g2ajt8UTJ80rYf0dajsHKt2VPmjsD2MXdql1NI/k4oposkFxBH1jJqibNdZYB2JK3Xaa W8fxUD4eCsoAHQBchl49THouhe4lEr4AE9QSVqBV1m7ksV/YqkllmYTcz1/Ytp2NJzEs MFu0nzRkGk6iGKouEIDDrMmrw8bmib4qkviIwpgZT+n+HDkWgQB4OjzfbZ1/EyLfiTNq VP7A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1713843680; x=1714448480; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=KkBGMpzb7Zyfmx1ZstJgk5Op8ktl/5dFs5p8PlYlyX8=; b=B8KRJmSEvpNTE3sc6UhzxgvdCzVznFVw/i5Y48w3HJ+lfLHEgCbZA1OORMKX+fM5Oj 2NYS8M0vXhyciT5QJdknis3RkrIjqAqtAjTvc37evFLqawtiDu4+6klcmvsQ69RX7UeT JWGwqjgtSSeX083nCWVBoUSfyDOLRXEZ2YkoFUH9ZzCoZApnYs2TQzNS2TcCKQlY7Njo hzllhMIpTbFI63ZRd29STBpITPEvQbU8Mxdu+4smZ81+jqCPxp0uNyyrdRAv5NPEhDHT kwqeFau63ddcLZXrXgZnVcGS9EzWx38uvIkfBMjOD80kGOysw4wIzivAUOv8ALJtpbmR coJg== X-Gm-Message-State: AOJu0Yz1OLD3yhtitOr6jWBB62iiDvPMMGOPxfD4mNVmBKhKm7Kspr4v niusKmOiEF7aFXjCdyuxSpyVecz1ATfzCGXTLEp194BDFMSbMLtXfAhmM0WwUoFe09jFUd8Rc2U 4 X-Google-Smtp-Source: AGHT+IFPWbPsOAgDjqPCg7khKs3ioBh3D8C//sMkXvFal/sejW6ukQN3/LZM7rjAVP3UsOTPrF/iBA== X-Received: by 2002:a05:6830:4392:b0:6ed:686:b43 with SMTP id s18-20020a056830439200b006ed06860b43mr9633062otv.17.1713843680244; Mon, 22 Apr 2024 20:41:20 -0700 (PDT) Received: from libai.bytedance.net ([61.213.176.11]) by smtp.gmail.com with ESMTPSA id b16-20020a63d810000000b005e438fe702dsm8266980pgh.65.2024.04.22.20.41.17 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 22 Apr 2024 20:41:19 -0700 (PDT) From: zhenwei pi To: linux-kernel@vger.kernel.org, linux-mm@kvack.org, virtualization@lists.linux.dev Cc: mst@redhat.com, david@redhat.com, jasowang@redhat.com, xuanzhuo@linux.alibaba.com, akpm@linux-foundation.org, zhenwei pi Subject: [PATCH v3 1/4] virtio_balloon: separate vm events into a function Date: Tue, 23 Apr 2024 11:41:06 +0800 Message-Id: <20240423034109.1552866-2-pizhenwei@bytedance.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240423034109.1552866-1-pizhenwei@bytedance.com> References: <20240423034109.1552866-1-pizhenwei@bytedance.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" All the VM events related statistics have dependence on 'CONFIG_VM_EVENT_COUNTERS', separate these events into a function to make code clean. Then we can remove 'CONFIG_VM_EVENT_COUNTERS' from 'update_balloon_stats'. Signed-off-by: zhenwei pi Reviewed-by: David Hildenbrand --- drivers/virtio/virtio_balloon.c | 43 ++++++++++++++++++++++----------- 1 file changed, 29 insertions(+), 14 deletions(-) diff --git a/drivers/virtio/virtio_balloon.c b/drivers/virtio/virtio_balloo= n.c index 1f5b3dd31fcf..1710e3098ecd 100644 --- a/drivers/virtio/virtio_balloon.c +++ b/drivers/virtio/virtio_balloon.c @@ -316,34 +316,49 @@ static inline void update_stat(struct virtio_balloon = *vb, int idx, =20 #define pages_to_bytes(x) ((u64)(x) << PAGE_SHIFT) =20 -static unsigned int update_balloon_stats(struct virtio_balloon *vb) +#ifdef CONFIG_VM_EVENT_COUNTERS +/* Return the number of entries filled by vm events */ +static inline unsigned int update_balloon_vm_stats(struct virtio_balloon *= vb) { unsigned long events[NR_VM_EVENT_ITEMS]; - struct sysinfo i; unsigned int idx =3D 0; - long available; - unsigned long caches; =20 all_vm_events(events); - si_meminfo(&i); - - available =3D si_mem_available(); - caches =3D global_node_page_state(NR_FILE_PAGES); - -#ifdef CONFIG_VM_EVENT_COUNTERS update_stat(vb, idx++, VIRTIO_BALLOON_S_SWAP_IN, - pages_to_bytes(events[PSWPIN])); + pages_to_bytes(events[PSWPIN])); update_stat(vb, idx++, VIRTIO_BALLOON_S_SWAP_OUT, - pages_to_bytes(events[PSWPOUT])); + pages_to_bytes(events[PSWPOUT])); update_stat(vb, idx++, VIRTIO_BALLOON_S_MAJFLT, events[PGMAJFAULT]); update_stat(vb, idx++, VIRTIO_BALLOON_S_MINFLT, events[PGFAULT]); + #ifdef CONFIG_HUGETLB_PAGE update_stat(vb, idx++, VIRTIO_BALLOON_S_HTLB_PGALLOC, events[HTLB_BUDDY_PGALLOC]); update_stat(vb, idx++, VIRTIO_BALLOON_S_HTLB_PGFAIL, events[HTLB_BUDDY_PGALLOC_FAIL]); -#endif -#endif +#endif /* CONFIG_HUGETLB_PAGE */ + + return idx; +} +#else /* CONFIG_VM_EVENT_COUNTERS */ +static inline unsigned int update_balloon_vm_stats(struct virtio_balloon *= vb) +{ + return 0; +} +#endif /* CONFIG_VM_EVENT_COUNTERS */ + +static unsigned int update_balloon_stats(struct virtio_balloon *vb) +{ + struct sysinfo i; + unsigned int idx; + long available; + unsigned long caches; + + idx =3D update_balloon_vm_stats(vb); + + si_meminfo(&i); + available =3D si_mem_available(); + caches =3D global_node_page_state(NR_FILE_PAGES); update_stat(vb, idx++, VIRTIO_BALLOON_S_MEMFREE, pages_to_bytes(i.freeram)); update_stat(vb, idx++, VIRTIO_BALLOON_S_MEMTOT, --=20 2.34.1 From nobody Mon Feb 9 17:58:11 2026 Received: from mail-io1-f50.google.com (mail-io1-f50.google.com [209.85.166.50]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9EFEC1F934 for ; Tue, 23 Apr 2024 03:41:24 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.166.50 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1713843686; cv=none; b=OsVIvCatpOjA7PcNwAYykjQS0CJtva3n8GMf/OAZQBZJX2Rx1fiaP+rnPStwytq6O3Xs5j53u/BelzzWfH5W+6CGBeoQnd1SITGYcn+90u0LtcrMAHE45+vgd39mWHJkP8buvJFGZxSBBDYJO0vN+Wssxlf9MuCe5MhxYdqA6Po= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1713843686; c=relaxed/simple; bh=og/1Bh1HmmM5su2/my/Vew9YvJYFlLIPs0D6nqUUBV4=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=Eqo+9DfKT/z2kEfyQIbqcPICAwaCx+r+k4YIiLu0vVvtdPzLTNau0i0rzM5k8RNPjtJveF97JEgjCfOekRCYdVGHvYRdc12N0IyspLoLIasqkvNrFevJ2UP6JISeQv03RTcFQUSTCgAuVbDDmcLQ8d4NfmTaMEXRWy2ZVdso5kE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com; spf=pass smtp.mailfrom=bytedance.com; dkim=pass (2048-bit key) header.d=bytedance.com header.i=@bytedance.com header.b=E2iiYPuH; arc=none smtp.client-ip=209.85.166.50 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=bytedance.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=bytedance.com header.i=@bytedance.com header.b="E2iiYPuH" Received: by mail-io1-f50.google.com with SMTP id ca18e2360f4ac-7da37436e36so204082139f.0 for ; Mon, 22 Apr 2024 20:41:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1713843683; x=1714448483; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=Zk8sHHlFZ5yMas0e7wt0QxJKNRueI84ut/8EKgUk1gU=; b=E2iiYPuHOseBUVaZBchysHsQOOYWR5njICBeqjKVeriE3PjZql5ialdTz8bcnINDo1 tjnwHTDcqZg9dMAq8g7S7UBVno4o2cc+I7XjsB77+x8lMlNpR0fR/yByjEozrT48GR4f CZf9zVV1aeNa+86l5sGAecCg6hc7zf8X5LqVTVDinUjLucteG8pyvWk3ai7QeJguop3c 0W3XX90KIM3abb5/eiW7B9hjHnu3tcyAXGJXDmR//cWEWthqSmHwPHCL3XQ2actqPe19 M3gs37Opg1rgoDVm6quTzcfnraZiCJikRQ/vi3EzKJiF1BEo1PQS7lWXSbS6tdJbyW+s S/xA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1713843683; x=1714448483; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Zk8sHHlFZ5yMas0e7wt0QxJKNRueI84ut/8EKgUk1gU=; b=sfSSq8/MRHuW/cUvbNDSDFRNYskVApr2gDIZf4FjUFjDLpQ0vnPeVBufOCAG2JTpu1 BF3rsFyyRtYTBFU6rou3kqQuuWJklx4Z6L3Fq1G9HU19E7bgSS9Vme3IaJ/EmlJnl3+P ka3CbUyDJ7eHIwNVt8GdL2hBbZwZ3mRWZVr96/ywpDzv7bDdsZBX3PTg+RzDX5s2RYIT 4UVrmN2VnUsaMqi2WLGEAEzrk9+xXEf52lI96aiiuUmJR24vd16c9q5GkHFmWMvk7gJC IIn/cvSnGqa/qb4ffTkbgj4+z0E65NK1bDz5mDTDKmBAk5BEq7aRV9dra3BTMb7mVuXe EovA== X-Gm-Message-State: AOJu0Yzz+gQG+3Sc/A+GY1DcpGSLYnXTlg7dHZ5T7UX2iXaqq+oUvQsT uAY5UmMwaGYRHL70K6e6dib8X/3ly6geEftflRVgAP7sO6utQQ1aOVnoklgg7NHibE2NI2u9/mt H X-Google-Smtp-Source: AGHT+IGK5wXI7LkTDRk5nDef0rXFGK962Vq/MSDSyQakfWBHz0PVFwhhTJdfJ5JmoY+i8cpaYiedhg== X-Received: by 2002:a92:c267:0:b0:36a:fe86:5fcc with SMTP id h7-20020a92c267000000b0036afe865fccmr16168240ild.0.1713843683557; Mon, 22 Apr 2024 20:41:23 -0700 (PDT) Received: from libai.bytedance.net ([61.213.176.11]) by smtp.gmail.com with ESMTPSA id b16-20020a63d810000000b005e438fe702dsm8266980pgh.65.2024.04.22.20.41.20 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 22 Apr 2024 20:41:23 -0700 (PDT) From: zhenwei pi To: linux-kernel@vger.kernel.org, linux-mm@kvack.org, virtualization@lists.linux.dev Cc: mst@redhat.com, david@redhat.com, jasowang@redhat.com, xuanzhuo@linux.alibaba.com, akpm@linux-foundation.org, zhenwei pi Subject: [PATCH v3 2/4] virtio_balloon: introduce oom-kill invocations Date: Tue, 23 Apr 2024 11:41:07 +0800 Message-Id: <20240423034109.1552866-3-pizhenwei@bytedance.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240423034109.1552866-1-pizhenwei@bytedance.com> References: <20240423034109.1552866-1-pizhenwei@bytedance.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" When the guest OS runs under critical memory pressure, the guest starts to kill processes. A guest monitor agent may scan 'oom_kill' from /proc/vmstat, and reports the OOM KILL event. However, the agent may be killed and we will loss this critical event(and the later events). For now we can also grep for magic words in guest kernel log from host side. Rather than this unstable way, virtio balloon reports OOM-KILL invocations instead. Acked-by: David Hildenbrand Signed-off-by: zhenwei pi --- drivers/virtio/virtio_balloon.c | 1 + include/uapi/linux/virtio_balloon.h | 6 ++++-- 2 files changed, 5 insertions(+), 2 deletions(-) diff --git a/drivers/virtio/virtio_balloon.c b/drivers/virtio/virtio_balloo= n.c index 1710e3098ecd..f7a47eaa0936 100644 --- a/drivers/virtio/virtio_balloon.c +++ b/drivers/virtio/virtio_balloon.c @@ -330,6 +330,7 @@ static inline unsigned int update_balloon_vm_stats(stru= ct virtio_balloon *vb) pages_to_bytes(events[PSWPOUT])); update_stat(vb, idx++, VIRTIO_BALLOON_S_MAJFLT, events[PGMAJFAULT]); update_stat(vb, idx++, VIRTIO_BALLOON_S_MINFLT, events[PGFAULT]); + update_stat(vb, idx++, VIRTIO_BALLOON_S_OOM_KILL, events[OOM_KILL]); =20 #ifdef CONFIG_HUGETLB_PAGE update_stat(vb, idx++, VIRTIO_BALLOON_S_HTLB_PGALLOC, diff --git a/include/uapi/linux/virtio_balloon.h b/include/uapi/linux/virti= o_balloon.h index ddaa45e723c4..b17bbe033697 100644 --- a/include/uapi/linux/virtio_balloon.h +++ b/include/uapi/linux/virtio_balloon.h @@ -71,7 +71,8 @@ struct virtio_balloon_config { #define VIRTIO_BALLOON_S_CACHES 7 /* Disk caches */ #define VIRTIO_BALLOON_S_HTLB_PGALLOC 8 /* Hugetlb page allocations */ #define VIRTIO_BALLOON_S_HTLB_PGFAIL 9 /* Hugetlb page allocation failu= res */ -#define VIRTIO_BALLOON_S_NR 10 +#define VIRTIO_BALLOON_S_OOM_KILL 10 /* OOM killer invocations */ +#define VIRTIO_BALLOON_S_NR 11 =20 #define VIRTIO_BALLOON_S_NAMES_WITH_PREFIX(VIRTIO_BALLOON_S_NAMES_prefix) = { \ VIRTIO_BALLOON_S_NAMES_prefix "swap-in", \ @@ -83,7 +84,8 @@ struct virtio_balloon_config { VIRTIO_BALLOON_S_NAMES_prefix "available-memory", \ VIRTIO_BALLOON_S_NAMES_prefix "disk-caches", \ VIRTIO_BALLOON_S_NAMES_prefix "hugetlb-allocations", \ - VIRTIO_BALLOON_S_NAMES_prefix "hugetlb-failures" \ + VIRTIO_BALLOON_S_NAMES_prefix "hugetlb-failures", \ + VIRTIO_BALLOON_S_NAMES_prefix "oom-kills" \ } =20 #define VIRTIO_BALLOON_S_NAMES VIRTIO_BALLOON_S_NAMES_WITH_PREFIX("") --=20 2.34.1 From nobody Mon Feb 9 17:58:11 2026 Received: from mail-ot1-f43.google.com (mail-ot1-f43.google.com [209.85.210.43]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 0457C20B20 for ; Tue, 23 Apr 2024 03:41:27 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.43 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1713843689; cv=none; b=avoZRemyFE/Efw6nZqSptAmPFEzMj7MpXznpWcVLJOYF3jeRm9huyohHICVdYcal6iq/QUD/F4Eb3neo7X40gn1XcxLuzozC5TIMZa3rscLXH5+lx91ByAQDp/Ebc3CL1tvbsjuZtVaBwhrtKN132dg6NC8hB/Bvempr9fBndz8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1713843689; c=relaxed/simple; bh=+kh9BcjI07go1nGvUEVdhO4Ly9aJPFfeZnFM+8cusGA=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=C7QM5w9nVqjmjAswLrlfJVCs+WhdUc0H9ZF2fhF47PVcORb1JiAbnQdXdmX8v8Eq/QzKVbZ8eeasnZKjF+6rhXnskoDKv0vNrf2VCCeCK/uhfkSjamz9pVJJgEtJGR3XQTwCLdgR7RZNqkNDlqLA/oNFsW7xOoe9NrZVASDpEsg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com; spf=pass smtp.mailfrom=bytedance.com; dkim=pass (2048-bit key) header.d=bytedance.com header.i=@bytedance.com header.b=NlXl/VP1; arc=none smtp.client-ip=209.85.210.43 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=bytedance.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=bytedance.com header.i=@bytedance.com header.b="NlXl/VP1" Received: by mail-ot1-f43.google.com with SMTP id 46e09a7af769-6eb55942409so2650918a34.1 for ; Mon, 22 Apr 2024 20:41:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1713843687; x=1714448487; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=F5+no2YCmidvNin275LSwR8pI0Tp0FoxLTpLeIvaXuY=; b=NlXl/VP1TLcShJ0LLkVd/JJa8QUjFpTdmVq7jxQAtx90vOJzMhnTlxTUI9gF5ouM6u 7rwz3/bgDySoqBDXegPdVpkm+pgStxxFeK7qs2FTiezRO8cocHuCu1Id5/46nygOhXRW 2O/FwYsf33BBF8JbL2beTKjcgfOlWjL+bzJ3bdAKDS/XUTdJKLd7q3uahZ/v7sPRGARi xAuKYJQBEvxeEKEcwc8bI09ItKNFOZBRR2ENlorM0z8IPN0vy9R1ONBawKrPIvNzq9Tv eLPsmimxP5CAnXzwkkb5okE7CKQ0ZMI0zNQQwtNfb0QbTnVST39cRaJUVMmFOp1ndzRV Bqwg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1713843687; x=1714448487; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=F5+no2YCmidvNin275LSwR8pI0Tp0FoxLTpLeIvaXuY=; b=sv+JYNxEVOG7mDTAnrtbFNChx7R+1NZjUitbtimd32o/mJhwYxW4UXhyruAtRZs/fx dZnZwdgyY/cGZ+SYN1eT41DjqdH0UDBySyazzfHQz4PDc4fhr0lcubNN7rih4DlUb2qp JwQPq+3/LlbpiJLdfQV5wqn/aep+D+Q1XHrKN5SSgCt4LwaiY5DHyk58nLgEAkYB4jfB 5+hoLUvYghU7x2c5SNot2UfBd4Yp51uZxbUwqxaevhIAKBD2fTmHvR3260i7HwaM2QQS pwQ5CvInHTPABN8LAnKk+rVQ7MqplLWm8e7cPLJOm529JIVmEQl++T3Aq9VSFL6u+v1w hy4g== X-Gm-Message-State: AOJu0YzbK02xvYW39MpKEpvVLcjXIhSzbQZSzRXLe15L9DBgxl5vBFAC exW001/x+MobquA2zNkXfujlmYfU8usuWLWKRluABZl0JGT+6QW/RuEdt+Vns4iz5wc8x0a/T60 9 X-Google-Smtp-Source: AGHT+IEk5ynVvllFpoDAECg8nh7BfbCqfpMIzwNakNzM2tHibdMZWatAJwp62coEKbLv8SpF4bdD4A== X-Received: by 2002:a05:6871:7a11:b0:22a:7899:37c4 with SMTP id pc17-20020a0568717a1100b0022a789937c4mr14499731oac.20.1713843686932; Mon, 22 Apr 2024 20:41:26 -0700 (PDT) Received: from libai.bytedance.net ([61.213.176.11]) by smtp.gmail.com with ESMTPSA id b16-20020a63d810000000b005e438fe702dsm8266980pgh.65.2024.04.22.20.41.23 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 22 Apr 2024 20:41:26 -0700 (PDT) From: zhenwei pi To: linux-kernel@vger.kernel.org, linux-mm@kvack.org, virtualization@lists.linux.dev Cc: mst@redhat.com, david@redhat.com, jasowang@redhat.com, xuanzhuo@linux.alibaba.com, akpm@linux-foundation.org, zhenwei pi Subject: [PATCH v3 3/4] virtio_balloon: introduce memory allocation stall counter Date: Tue, 23 Apr 2024 11:41:08 +0800 Message-Id: <20240423034109.1552866-4-pizhenwei@bytedance.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240423034109.1552866-1-pizhenwei@bytedance.com> References: <20240423034109.1552866-1-pizhenwei@bytedance.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Memory allocation stall counter represents the performance/latency of memory allocation, expose this counter to the host side by virtio balloon device via out-of-bound way. Acked-by: David Hildenbrand Signed-off-by: zhenwei pi --- drivers/virtio/virtio_balloon.c | 8 ++++++++ include/uapi/linux/virtio_balloon.h | 6 ++++-- 2 files changed, 12 insertions(+), 2 deletions(-) diff --git a/drivers/virtio/virtio_balloon.c b/drivers/virtio/virtio_balloo= n.c index f7a47eaa0936..e6229e548832 100644 --- a/drivers/virtio/virtio_balloon.c +++ b/drivers/virtio/virtio_balloon.c @@ -322,6 +322,8 @@ static inline unsigned int update_balloon_vm_stats(stru= ct virtio_balloon *vb) { unsigned long events[NR_VM_EVENT_ITEMS]; unsigned int idx =3D 0; + unsigned int zid; + unsigned long stall =3D 0; =20 all_vm_events(events); update_stat(vb, idx++, VIRTIO_BALLOON_S_SWAP_IN, @@ -332,6 +334,12 @@ static inline unsigned int update_balloon_vm_stats(str= uct virtio_balloon *vb) update_stat(vb, idx++, VIRTIO_BALLOON_S_MINFLT, events[PGFAULT]); update_stat(vb, idx++, VIRTIO_BALLOON_S_OOM_KILL, events[OOM_KILL]); =20 + /* sum all the stall events */ + for (zid =3D 0; zid < MAX_NR_ZONES; zid++) + stall +=3D events[ALLOCSTALL_NORMAL - ZONE_NORMAL + zid]; + + update_stat(vb, idx++, VIRTIO_BALLOON_S_ALLOC_STALL, stall); + #ifdef CONFIG_HUGETLB_PAGE update_stat(vb, idx++, VIRTIO_BALLOON_S_HTLB_PGALLOC, events[HTLB_BUDDY_PGALLOC]); diff --git a/include/uapi/linux/virtio_balloon.h b/include/uapi/linux/virti= o_balloon.h index b17bbe033697..487b893a160e 100644 --- a/include/uapi/linux/virtio_balloon.h +++ b/include/uapi/linux/virtio_balloon.h @@ -72,7 +72,8 @@ struct virtio_balloon_config { #define VIRTIO_BALLOON_S_HTLB_PGALLOC 8 /* Hugetlb page allocations */ #define VIRTIO_BALLOON_S_HTLB_PGFAIL 9 /* Hugetlb page allocation failu= res */ #define VIRTIO_BALLOON_S_OOM_KILL 10 /* OOM killer invocations */ -#define VIRTIO_BALLOON_S_NR 11 +#define VIRTIO_BALLOON_S_ALLOC_STALL 11 /* Stall count of memory allocat= oin */ +#define VIRTIO_BALLOON_S_NR 12 =20 #define VIRTIO_BALLOON_S_NAMES_WITH_PREFIX(VIRTIO_BALLOON_S_NAMES_prefix) = { \ VIRTIO_BALLOON_S_NAMES_prefix "swap-in", \ @@ -85,7 +86,8 @@ struct virtio_balloon_config { VIRTIO_BALLOON_S_NAMES_prefix "disk-caches", \ VIRTIO_BALLOON_S_NAMES_prefix "hugetlb-allocations", \ VIRTIO_BALLOON_S_NAMES_prefix "hugetlb-failures", \ - VIRTIO_BALLOON_S_NAMES_prefix "oom-kills" \ + VIRTIO_BALLOON_S_NAMES_prefix "oom-kills", \ + VIRTIO_BALLOON_S_NAMES_prefix "alloc-stalls" \ } =20 #define VIRTIO_BALLOON_S_NAMES VIRTIO_BALLOON_S_NAMES_WITH_PREFIX("") --=20 2.34.1 From nobody Mon Feb 9 17:58:11 2026 Received: from mail-pf1-f177.google.com (mail-pf1-f177.google.com [209.85.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 64683224EF for ; Tue, 23 Apr 2024 03:41:31 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.177 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1713843692; cv=none; b=s9NyCWg0C0rnHeCP97rjUNepn2DuJJHZnHs5TAXlowED0g25vsWRzgaM8a4YRuK8pSHckORdT6ApGIAX+PF1bM6g6Nkx+h/YJjIUVHQQbeRN3UqYfSuYKbeEF9SR8nn8QFSWlq4xzHMcPiBYOepmmIZghpSwNB7QuEnnsGEzJDA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1713843692; c=relaxed/simple; bh=T2mh8rRfGjxk++iQjwyZbxgJDDwtHaHq4Lo1WcLQsh8=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=kgnpSk6upBUDAgQTzamB9n033ArX357MJ7rGiDQotRaZsr5hVRKxRo2qfmmO/cQ4nx6jPEPf/sX6ZKUBuiz6u/mfi7SadkMbWbgM2amgOWKVIcp6tUgXKIA1LUxAOhwf1lGC1Ay+jHpF+w6sq+bMQW8cPJZu21dN8w2+xLjvwVA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com; spf=pass smtp.mailfrom=bytedance.com; dkim=pass (2048-bit key) header.d=bytedance.com header.i=@bytedance.com header.b=Sf7wDfjX; arc=none smtp.client-ip=209.85.210.177 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=bytedance.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=bytedance.com header.i=@bytedance.com header.b="Sf7wDfjX" Received: by mail-pf1-f177.google.com with SMTP id d2e1a72fcca58-6ee0642f718so4301848b3a.0 for ; Mon, 22 Apr 2024 20:41:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1713843690; x=1714448490; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=P0uuoQxJ0kINjaQgEP7wqVGQ+5rr3s87tkW4G+81Ziw=; b=Sf7wDfjXTjkUIcH39MDLDCX6rPMe7ProY+HtT7tUlsJ59wnIxBr7OCMPfIR2ZQkF/q HPRvZYtyJjOR017eFw0SNYGHHNuDNGdjRylt/8gZA2KfGNgVkky26CbrteXMW8g62EsY HyzBgDL5esMWYiaNQsdlBcawBZFiLzjJg2jISCylaccXQ/nlEm2GoifQ8uZ64jWPyh5Q 6DszFf9rUiWYMunZF4j0yMjbcvhmol38XY6wMkdt7bilZt0k7go/1pDBImalHqB4qETE Y344O+5eEO1+A1+OMieuvPTt9GNXM5ZpFfubF3SOlmunf3+Zvm3jmopxkSa1TEus7asO DBLg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1713843690; x=1714448490; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=P0uuoQxJ0kINjaQgEP7wqVGQ+5rr3s87tkW4G+81Ziw=; b=nOwfdz3KrzP84tYVVtjWEwdZF0tVyzxt/jAD1WqWUMXJBuO40lRyVj+YYPHtFlcq2F RF99S5icNyZPqk7HXZK/MCgdkcUbfu5Lu+auYVGrboMJd8SdVcYQReQdXlsh4ge8F3gx RisbrS4yV3kNNmG9ylwSdqsw5LWsKU5uY1FHNJ4MFil2rbWWBQcirGOdFYZiJu3kI4OA kMMdc5sz60+92oYL/ijPndCnfwUdzhCIQKLjW2/yIt0fGzZmwfk/u2krRhYUwkrrxfxv newl+Mr7ew0ebXlrVtj0SSH9UmrfAEL6dnErp/H0I4cMUnPAfsFeH4Ao42yITILA8IqM LHTg== X-Gm-Message-State: AOJu0YweFG7CHUmMmtdF09cb2Wboj9LBtyfn9gMEB4+f1bXwmKQurX1+ 07c53mPIZY7tPQx9Wyuy9B/zIhCVQxqsiVLeHWkd8Df0jQbF0SrIwdmWiieemVKnMdDSSma84aY J X-Google-Smtp-Source: AGHT+IH3OpN2sFYi3GvJfqyxhabzuW3WUwrv+oQUG4aXzn7ZEeGp3bOQixCyYcifZfO95/rYrfTxJA== X-Received: by 2002:a05:6a20:841b:b0:1a9:54fe:b9b7 with SMTP id c27-20020a056a20841b00b001a954feb9b7mr2176395pzd.21.1713843690300; Mon, 22 Apr 2024 20:41:30 -0700 (PDT) Received: from libai.bytedance.net ([61.213.176.11]) by smtp.gmail.com with ESMTPSA id b16-20020a63d810000000b005e438fe702dsm8266980pgh.65.2024.04.22.20.41.27 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 22 Apr 2024 20:41:30 -0700 (PDT) From: zhenwei pi To: linux-kernel@vger.kernel.org, linux-mm@kvack.org, virtualization@lists.linux.dev Cc: mst@redhat.com, david@redhat.com, jasowang@redhat.com, xuanzhuo@linux.alibaba.com, akpm@linux-foundation.org, zhenwei pi Subject: [PATCH v3 4/4] virtio_balloon: introduce memory scan/reclaim info Date: Tue, 23 Apr 2024 11:41:09 +0800 Message-Id: <20240423034109.1552866-5-pizhenwei@bytedance.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240423034109.1552866-1-pizhenwei@bytedance.com> References: <20240423034109.1552866-1-pizhenwei@bytedance.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Expose memory scan/reclaim information to the host side via virtio balloon device. Now we have a metric to analyze the memory performance: y: counter increases n: counter does not changes h: the rate of counter change is high l: the rate of counter change is low OOM: VIRTIO_BALLOON_S_OOM_KILL STALL: VIRTIO_BALLOON_S_ALLOC_STALL ASCAN: VIRTIO_BALLOON_S_SCAN_ASYNC DSCAN: VIRTIO_BALLOON_S_SCAN_DIRECT ARCLM: VIRTIO_BALLOON_S_RECLAIM_ASYNC DRCLM: VIRTIO_BALLOON_S_RECLAIM_DIRECT - OOM[y], STALL[*], ASCAN[*], DSCAN[*], ARCLM[*], DRCLM[*]: the guest runs under really critial memory pressure - OOM[n], STALL[h], ASCAN[*], DSCAN[l], ARCLM[*], DRCLM[l]: the memory allocation stalls due to cgroup, not the global memory pressure. - OOM[n], STALL[h], ASCAN[*], DSCAN[h], ARCLM[*], DRCLM[h]: the memory allocation stalls due to global memory pressure. The performance gets hurt a lot. A high ratio between DRCLM/DSCAN shows quite effective memory reclaiming. - OOM[n], STALL[h], ASCAN[*], DSCAN[h], ARCLM[*], DRCLM[l]: the memory allocation stalls due to global memory pressure. the ratio between DRCLM/DSCAN gets low, the guest OS is thrashing heavily, the serious case leads poor performance and difficult trouble shooting. Ex, sshd may block on memory allocation when accepting new connections, a user can't login a VM by ssh command. - OOM[n], STALL[n], ASCAN[h], DSCAN[n], ARCLM[l], DRCLM[n]: the low ratio between ARCLM/ASCAN shows that the guest tries to reclaim more memory, but it can't. Once more memory is required in future, it will struggle to reclaim memory. Acked-by: David Hildenbrand Signed-off-by: zhenwei pi --- drivers/virtio/virtio_balloon.c | 9 +++++++++ include/uapi/linux/virtio_balloon.h | 12 ++++++++++-- 2 files changed, 19 insertions(+), 2 deletions(-) diff --git a/drivers/virtio/virtio_balloon.c b/drivers/virtio/virtio_balloo= n.c index e6229e548832..225662358221 100644 --- a/drivers/virtio/virtio_balloon.c +++ b/drivers/virtio/virtio_balloon.c @@ -340,6 +340,15 @@ static inline unsigned int update_balloon_vm_stats(str= uct virtio_balloon *vb) =20 update_stat(vb, idx++, VIRTIO_BALLOON_S_ALLOC_STALL, stall); =20 + update_stat(vb, idx++, VIRTIO_BALLOON_S_ASYNC_SCAN, + pages_to_bytes(events[PGSCAN_KSWAPD])); + update_stat(vb, idx++, VIRTIO_BALLOON_S_DIRECT_SCAN, + pages_to_bytes(events[PGSCAN_DIRECT])); + update_stat(vb, idx++, VIRTIO_BALLOON_S_ASYNC_RECLAIM, + pages_to_bytes(events[PGSTEAL_KSWAPD])); + update_stat(vb, idx++, VIRTIO_BALLOON_S_DIRECT_RECLAIM, + pages_to_bytes(events[PGSTEAL_DIRECT])); + #ifdef CONFIG_HUGETLB_PAGE update_stat(vb, idx++, VIRTIO_BALLOON_S_HTLB_PGALLOC, events[HTLB_BUDDY_PGALLOC]); diff --git a/include/uapi/linux/virtio_balloon.h b/include/uapi/linux/virti= o_balloon.h index 487b893a160e..ee35a372805d 100644 --- a/include/uapi/linux/virtio_balloon.h +++ b/include/uapi/linux/virtio_balloon.h @@ -73,7 +73,11 @@ struct virtio_balloon_config { #define VIRTIO_BALLOON_S_HTLB_PGFAIL 9 /* Hugetlb page allocation failu= res */ #define VIRTIO_BALLOON_S_OOM_KILL 10 /* OOM killer invocations */ #define VIRTIO_BALLOON_S_ALLOC_STALL 11 /* Stall count of memory allocat= oin */ -#define VIRTIO_BALLOON_S_NR 12 +#define VIRTIO_BALLOON_S_ASYNC_SCAN 12 /* Amount of memory scanned asyn= chronously */ +#define VIRTIO_BALLOON_S_DIRECT_SCAN 13 /* Amount of memory scanned dire= ctly */ +#define VIRTIO_BALLOON_S_ASYNC_RECLAIM 14 /* Amount of memory reclaimed as= ynchronously */ +#define VIRTIO_BALLOON_S_DIRECT_RECLAIM 15 /* Amount of memory reclaimed d= irectly */ +#define VIRTIO_BALLOON_S_NR 16 =20 #define VIRTIO_BALLOON_S_NAMES_WITH_PREFIX(VIRTIO_BALLOON_S_NAMES_prefix) = { \ VIRTIO_BALLOON_S_NAMES_prefix "swap-in", \ @@ -87,7 +91,11 @@ struct virtio_balloon_config { VIRTIO_BALLOON_S_NAMES_prefix "hugetlb-allocations", \ VIRTIO_BALLOON_S_NAMES_prefix "hugetlb-failures", \ VIRTIO_BALLOON_S_NAMES_prefix "oom-kills", \ - VIRTIO_BALLOON_S_NAMES_prefix "alloc-stalls" \ + VIRTIO_BALLOON_S_NAMES_prefix "alloc-stalls", \ + VIRTIO_BALLOON_S_NAMES_prefix "async-scans", \ + VIRTIO_BALLOON_S_NAMES_prefix "direct-scans", \ + VIRTIO_BALLOON_S_NAMES_prefix "async-reclaims", \ + VIRTIO_BALLOON_S_NAMES_prefix "direct-reclaims" \ } =20 #define VIRTIO_BALLOON_S_NAMES VIRTIO_BALLOON_S_NAMES_WITH_PREFIX("") --=20 2.34.1