From nobody Wed Feb 11 04:00:54 2026 Received: from mail-pl1-f176.google.com (mail-pl1-f176.google.com [209.85.214.176]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E82044D9EA for ; Mon, 22 Apr 2024 07:43:05 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.176 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1713771788; cv=none; b=Cr3nxJrFv9Qv7NQF4CcurfAf6DJLhQ65xi7ReyThljSWaX9CeGKiPgdWNfhdDnrlibKHOCWW2mXM+cb23BcOENqhTd0CGj1KZXlhhOw4HDwetofOkDhknxSTDaTrgOrQeNLGhgDN08vTo9hgdyNNeYx6ea6JphrOCxVZsWSgJVY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1713771788; c=relaxed/simple; bh=3Oj9v8fKAeuQ0Snq0CF75spBnzuIUdmlULeL8Toft80=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=Ae7iNt5aqaifVU/bzAjXxRygwuJqprN3FdThIdQvcEt04m+HxAzTCvXW1VH8N8Atik6RTVd1ADwVn9H2QnU9b27Nte4Xu8lhes067IhkNzpk/G9FqiJ5RQwDyrymfiD5x9p7O4j2cgAFiFYncCBJQuW3/Wchwz0G5564+kMZKak= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com; spf=pass smtp.mailfrom=bytedance.com; dkim=pass (2048-bit key) header.d=bytedance.com header.i=@bytedance.com header.b=cxDrZ2ES; arc=none smtp.client-ip=209.85.214.176 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=bytedance.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=bytedance.com header.i=@bytedance.com header.b="cxDrZ2ES" Received: by mail-pl1-f176.google.com with SMTP id d9443c01a7336-1e4bf0b3e06so39388395ad.1 for ; Mon, 22 Apr 2024 00:43:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1713771785; x=1714376585; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=8v4Ayhiua6xBQPRLdZqElV8SuuukGuSIp7AWQ+b83kM=; b=cxDrZ2ESNLuYQ0nbcTG/tvKkWormhzfi/7K/8sRe1STdM5sum1C5ZUCzela5AOeU7c hdCFydD/Cf/FqfQVn0m1RMUEOcuhocTC7u0pZRnMmS5cAG9JgBZXIK3RUlk/t/Pfw3iJ XRQtXbizGEzIW+42qewepdMEBBlcPUyNru2RAleneqCijkdZvgOKLVmWwnOJksCRmu35 dw1Cy7SPXDv7n2nMDJ6k6hHqLHCuPRI1zJYfqnmDSWPAscqWuHQ7D7+2XVkn0aDyU2WI Lam6Rf4Cz6suMvNZ0MIMJtiNqoFIqS2xJ4sOnc61tvDuhMRPGsh0wXLqkUYjTCAXJzxX LHtw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1713771785; x=1714376585; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=8v4Ayhiua6xBQPRLdZqElV8SuuukGuSIp7AWQ+b83kM=; b=NjuQdN1sve4m7Obq4vCqcux7JauugeLEIcsjw2v6ZMaORJqUTaPzXN3zoKAYGMtIg5 fk2W3xKYQyNIMc+oGnLWqqT00/P4YV3cUvrODkMMPJ7cTWHKldFsMtuWl/5bF+kMYAhZ XR4u/xAAlIslVmAkxoZy3HMCI0o2rraxssv3xUuoHTjJs8yhUWY8xlqhPVLdzH9jGEQ6 QdzRUdD+QDkOaxc0xnu86oB2+wQ+7qh7g0tejKSIZXp2cIrfWAneqXZvoHHzsyjyXkFg dqjMTmnuKh7JipQ+sYn9MFiJ16650nD616wXOwagk6G9HU69V41p7tv3PB6S/hw459tS hwsg== X-Gm-Message-State: AOJu0YwsoJwPKi99VwE8/V1VT4IubBxse3+yarWtnI5/kkS2iKl/Z5bV FirWU51K5zbS19jQ5zgwx0dFlkKzYwzDWRr0ubH9yZAbo1j3BUnAhd+XbSt82Y/CkVXCsf7gFNR 1 X-Google-Smtp-Source: AGHT+IH5FapWr2NsqZRBiIrj5e5+IahI/4Uza3UkvKorGF9IdyGMaix/CVDPSu/65IlIfj7nO5mHKw== X-Received: by 2002:a17:902:c40c:b0:1e4:2b90:7282 with SMTP id k12-20020a170902c40c00b001e42b907282mr13198963plk.14.1713771785215; Mon, 22 Apr 2024 00:43:05 -0700 (PDT) Received: from libai.bytedance.net ([61.213.176.11]) by smtp.gmail.com with ESMTPSA id e11-20020a170902784b00b001e0bae4490fsm7595634pln.154.2024.04.22.00.43.02 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 22 Apr 2024 00:43:04 -0700 (PDT) From: zhenwei pi To: linux-kernel@vger.kernel.org, linux-mm@kvack.org, virtualization@lists.linux.dev Cc: mst@redhat.com, david@redhat.com, jasowang@redhat.com, xuanzhuo@linux.alibaba.com, akpm@linux-foundation.org, zhenwei pi Subject: [PATCH v2 1/4] virtio_balloon: separate vm events into a function Date: Mon, 22 Apr 2024 15:42:51 +0800 Message-Id: <20240422074254.1440457-2-pizhenwei@bytedance.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240422074254.1440457-1-pizhenwei@bytedance.com> References: <20240422074254.1440457-1-pizhenwei@bytedance.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" All the VM events related statistics have dependence on 'CONFIG_VM_EVENT_COUNTERS', once any stack variable is required by any VM events in future, we would have codes like: #ifdef CONFIG_VM_EVENT_COUNTERS unsigned long foo; #endif ... #ifdef CONFIG_VM_EVENT_COUNTERS foo =3D events[XXX] + events[YYY]; update_stat(vb, idx++, VIRTIO_BALLOON_S_XXX, foo); #endif Separate vm events into a single function, also remove 'CONFIG_VM_EVENT_COUNTERS' from 'update_balloon_stats'. Signed-off-by: zhenwei pi --- drivers/virtio/virtio_balloon.c | 44 ++++++++++++++++++++++----------- 1 file changed, 29 insertions(+), 15 deletions(-) diff --git a/drivers/virtio/virtio_balloon.c b/drivers/virtio/virtio_balloo= n.c index 1f5b3dd31fcf..59fe157e5722 100644 --- a/drivers/virtio/virtio_balloon.c +++ b/drivers/virtio/virtio_balloon.c @@ -316,34 +316,48 @@ static inline void update_stat(struct virtio_balloon = *vb, int idx, =20 #define pages_to_bytes(x) ((u64)(x) << PAGE_SHIFT) =20 -static unsigned int update_balloon_stats(struct virtio_balloon *vb) +/* Return the number of entries filled by vm events */ +static inline unsigned int update_balloon_vm_stats(struct virtio_balloon *= vb, + unsigned int start) { +#ifdef CONFIG_VM_EVENT_COUNTERS unsigned long events[NR_VM_EVENT_ITEMS]; - struct sysinfo i; - unsigned int idx =3D 0; - long available; - unsigned long caches; + unsigned int idx =3D start; =20 all_vm_events(events); - si_meminfo(&i); - - available =3D si_mem_available(); - caches =3D global_node_page_state(NR_FILE_PAGES); - -#ifdef CONFIG_VM_EVENT_COUNTERS update_stat(vb, idx++, VIRTIO_BALLOON_S_SWAP_IN, - pages_to_bytes(events[PSWPIN])); + pages_to_bytes(events[PSWPIN])); update_stat(vb, idx++, VIRTIO_BALLOON_S_SWAP_OUT, - pages_to_bytes(events[PSWPOUT])); + pages_to_bytes(events[PSWPOUT])); update_stat(vb, idx++, VIRTIO_BALLOON_S_MAJFLT, events[PGMAJFAULT]); update_stat(vb, idx++, VIRTIO_BALLOON_S_MINFLT, events[PGFAULT]); + #ifdef CONFIG_HUGETLB_PAGE update_stat(vb, idx++, VIRTIO_BALLOON_S_HTLB_PGALLOC, events[HTLB_BUDDY_PGALLOC]); update_stat(vb, idx++, VIRTIO_BALLOON_S_HTLB_PGFAIL, events[HTLB_BUDDY_PGALLOC_FAIL]); -#endif -#endif +#endif /* CONFIG_HUGETLB_PAGE */ + + return idx - start; +#else /* CONFIG_VM_EVENT_COUNTERS */ + + return 0; +#endif /* CONFIG_VM_EVENT_COUNTERS */ +} + +static unsigned int update_balloon_stats(struct virtio_balloon *vb) +{ + struct sysinfo i; + unsigned int idx =3D 0; + long available; + unsigned long caches; + + idx +=3D update_balloon_vm_stats(vb, idx); + + si_meminfo(&i); + available =3D si_mem_available(); + caches =3D global_node_page_state(NR_FILE_PAGES); update_stat(vb, idx++, VIRTIO_BALLOON_S_MEMFREE, pages_to_bytes(i.freeram)); update_stat(vb, idx++, VIRTIO_BALLOON_S_MEMTOT, --=20 2.34.1 From nobody Wed Feb 11 04:00:54 2026 Received: from mail-pf1-f170.google.com (mail-pf1-f170.google.com [209.85.210.170]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B107B4F890 for ; Mon, 22 Apr 2024 07:43:09 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.170 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1713771791; cv=none; b=hpCvvty7XTifWHR5T4GGSVehX31IegoWLKwvm6gA0CzV6KRXmtEya3UdaboPrEt5zbl/WqBa7qW51Qocbvb4oN7MFcq81wRMYHjXgBB1Oa4rzAFpCWSa+YGeYPJ+O1hRhvPQjKQqE8jRCKyZ4qPeyAhV7OElP0xbexRnAWag7NY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1713771791; c=relaxed/simple; bh=ITAI70NjeaKrkJrIc+02MfHU2bt8vzw+2K++tUngTyY=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=BE9MTo4VdcgN2Xdp0YHiLVFxZsopP1TaXs3ZEvj6F3dE7Y5EgFV3VC7za0wTuYZme5+NlYLFT9yzZZJyQ964MFgrYBCvwUx49muGXnUSwXLXVCEgFKXXbJKmXBucoGzCGW3wbpZLu5HUwsHKEleKeTI2iTI/UJ+1NTtzj+NOw3I= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com; spf=pass smtp.mailfrom=bytedance.com; dkim=pass (2048-bit key) header.d=bytedance.com header.i=@bytedance.com header.b=Wey4fnN9; arc=none smtp.client-ip=209.85.210.170 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=bytedance.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=bytedance.com header.i=@bytedance.com header.b="Wey4fnN9" Received: by mail-pf1-f170.google.com with SMTP id d2e1a72fcca58-6edc61d0ff6so4069007b3a.2 for ; Mon, 22 Apr 2024 00:43:09 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1713771789; x=1714376589; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=9/EWE7HosrBQLSJgBRmTeatXRmpZ6JVLuxoFHN85xXc=; b=Wey4fnN9UJ3PL6Ytmz+UWu5Il3DHOpmr76m4jpQeXlHpUyZa7ZOCFaRKdAgGQu7li8 n/5vdnL1Y5+fFaRmcrK/1nBn5/KycTEZTu36oi++kuT7i6feRYBRYSf7BcK5N+4QItBT l5RbzRu/QmZxZBTICZcqtksFlVMMpWgQDNcjfW/dUVEu8nwofLNfdz/hnt45o53jNLMc Nn6CYnmDMXXixtP/uBiV8mK7hMN0ZMTkMgxXmAaXPu/1q6kC/FblqfW8OMDs23J/uKzM Hwt7R6i0eT1BQLe10n1NsUK2UL6Iwlm5p25+h0jsRxUHt//efRx0dNCFG+RXBAPNy9Ga t9Tw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1713771789; x=1714376589; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=9/EWE7HosrBQLSJgBRmTeatXRmpZ6JVLuxoFHN85xXc=; b=PTF3X/Zom2kEVNw0II+pHB6ADImLBLHhke56HPH0BLF7sM8UE9HQqd5uInExucKGPD yUzlDaMqfdmmR2SprGokXkdiLUSP9C2teoEQrIlT/OUqsWqw3iynLfW8Rw0NBEEcr5Qo h0US6xTrHQmSCA3jXgRyrGc4PEVO8dOKVZWyYdmzfYKJwg78B9TG25dITbsz5PlOn2KC Xx97ZLmaGDWFKwmj4hxN2co67ga2xZkIyPOxXTTk2l5QdvpHQr8L0nV4YSrLT5mUZCVW qnZPPnSizgzbgJyUOUUGJM1yAzonv/udgtgEFiVD1Nr9URZIXsyssWqPa/ZNk0NzMItV kYXQ== X-Gm-Message-State: AOJu0YzEPgSJ9gVmqPGwWwW1x6+JdY9riePBxALxQcBTiJnN1uPXtmk9 WEJ3TVq2kAlxjrDTAjhMZk8l7NnDJ4AD342rd48BQxvpa0LQF56BvAOT1iX3Sr6dcgWe1QI7l2i X X-Google-Smtp-Source: AGHT+IE2E7bgMyLmnmQSu1kHHGwGpjyE4p40kY0+jyTlKg2yf/aOeq6zrS4zsbnrUiBWDAWUZBb4Rw== X-Received: by 2002:a05:6a21:348b:b0:1ac:3a2c:b42 with SMTP id yo11-20020a056a21348b00b001ac3a2c0b42mr12591466pzb.9.1713771788742; Mon, 22 Apr 2024 00:43:08 -0700 (PDT) Received: from libai.bytedance.net ([61.213.176.11]) by smtp.gmail.com with ESMTPSA id e11-20020a170902784b00b001e0bae4490fsm7595634pln.154.2024.04.22.00.43.05 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 22 Apr 2024 00:43:08 -0700 (PDT) From: zhenwei pi To: linux-kernel@vger.kernel.org, linux-mm@kvack.org, virtualization@lists.linux.dev Cc: mst@redhat.com, david@redhat.com, jasowang@redhat.com, xuanzhuo@linux.alibaba.com, akpm@linux-foundation.org, zhenwei pi Subject: [PATCH v2 2/4] virtio_balloon: introduce oom-kill invocations Date: Mon, 22 Apr 2024 15:42:52 +0800 Message-Id: <20240422074254.1440457-3-pizhenwei@bytedance.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240422074254.1440457-1-pizhenwei@bytedance.com> References: <20240422074254.1440457-1-pizhenwei@bytedance.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" When the guest OS runs under critical memory pressure, the guest starts to kill processes. A guest monitor agent may scan 'oom_kill' from /proc/vmstat, and reports the OOM KILL event. However, the agent may be killed and we will loss this critical event(and the later events). For now we can also grep for magic words in guest kernel log from host side. Rather than this unstable way, virtio balloon reports OOM-KILL invocations instead. Acked-by: David Hildenbrand Signed-off-by: zhenwei pi --- drivers/virtio/virtio_balloon.c | 1 + include/uapi/linux/virtio_balloon.h | 6 ++++-- 2 files changed, 5 insertions(+), 2 deletions(-) diff --git a/drivers/virtio/virtio_balloon.c b/drivers/virtio/virtio_balloo= n.c index 59fe157e5722..87a1d6fa77fb 100644 --- a/drivers/virtio/virtio_balloon.c +++ b/drivers/virtio/virtio_balloon.c @@ -331,6 +331,7 @@ static inline unsigned int update_balloon_vm_stats(stru= ct virtio_balloon *vb, pages_to_bytes(events[PSWPOUT])); update_stat(vb, idx++, VIRTIO_BALLOON_S_MAJFLT, events[PGMAJFAULT]); update_stat(vb, idx++, VIRTIO_BALLOON_S_MINFLT, events[PGFAULT]); + update_stat(vb, idx++, VIRTIO_BALLOON_S_OOM_KILL, events[OOM_KILL]); =20 #ifdef CONFIG_HUGETLB_PAGE update_stat(vb, idx++, VIRTIO_BALLOON_S_HTLB_PGALLOC, diff --git a/include/uapi/linux/virtio_balloon.h b/include/uapi/linux/virti= o_balloon.h index ddaa45e723c4..b17bbe033697 100644 --- a/include/uapi/linux/virtio_balloon.h +++ b/include/uapi/linux/virtio_balloon.h @@ -71,7 +71,8 @@ struct virtio_balloon_config { #define VIRTIO_BALLOON_S_CACHES 7 /* Disk caches */ #define VIRTIO_BALLOON_S_HTLB_PGALLOC 8 /* Hugetlb page allocations */ #define VIRTIO_BALLOON_S_HTLB_PGFAIL 9 /* Hugetlb page allocation failu= res */ -#define VIRTIO_BALLOON_S_NR 10 +#define VIRTIO_BALLOON_S_OOM_KILL 10 /* OOM killer invocations */ +#define VIRTIO_BALLOON_S_NR 11 =20 #define VIRTIO_BALLOON_S_NAMES_WITH_PREFIX(VIRTIO_BALLOON_S_NAMES_prefix) = { \ VIRTIO_BALLOON_S_NAMES_prefix "swap-in", \ @@ -83,7 +84,8 @@ struct virtio_balloon_config { VIRTIO_BALLOON_S_NAMES_prefix "available-memory", \ VIRTIO_BALLOON_S_NAMES_prefix "disk-caches", \ VIRTIO_BALLOON_S_NAMES_prefix "hugetlb-allocations", \ - VIRTIO_BALLOON_S_NAMES_prefix "hugetlb-failures" \ + VIRTIO_BALLOON_S_NAMES_prefix "hugetlb-failures", \ + VIRTIO_BALLOON_S_NAMES_prefix "oom-kills" \ } =20 #define VIRTIO_BALLOON_S_NAMES VIRTIO_BALLOON_S_NAMES_WITH_PREFIX("") --=20 2.34.1 From nobody Wed Feb 11 04:00:54 2026 Received: from mail-pl1-f169.google.com (mail-pl1-f169.google.com [209.85.214.169]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id EC3CF502A4 for ; Mon, 22 Apr 2024 07:43:12 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.169 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1713771794; cv=none; b=YKiLS1zcRMcUPrJO4k2GNqCmDpypFXkK5VHcu87+N3JsYpdQKBWLA2YITbXP+vpaLjhQKZmjKGnrEU2gurrBWrah143symwUIEcKG3u3ahh3TmMsIO9uaUS8STV/CKofY60htFvpugZUvI65hLF7FVD252pUCknx6WBmxvUt8oA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1713771794; c=relaxed/simple; bh=Tifl00pi8az9qD8WZxeEmmg/browFVnK/hcBZYXoc2g=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=uMPuZmtHFfULP5xXzQJCG/A0A6bPdqtPCRty6oIH6wyFr3nSf3BCm/9cvjX+pkOpPgDXEjVMf1tHbSm8xqMQN0IKGto0Q5FFQilYT5IDmjagvpf9i4xv+ldcM+8wjZPcIxUzkbMT7Iz7yKtaULXHg82uaesQcxEwIIXkqfIlOmU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com; spf=pass smtp.mailfrom=bytedance.com; dkim=pass (2048-bit key) header.d=bytedance.com header.i=@bytedance.com header.b=TajcPByk; arc=none smtp.client-ip=209.85.214.169 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=bytedance.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=bytedance.com header.i=@bytedance.com header.b="TajcPByk" Received: by mail-pl1-f169.google.com with SMTP id d9443c01a7336-1e2bbc2048eso33640975ad.3 for ; Mon, 22 Apr 2024 00:43:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1713771792; x=1714376592; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=zrl4akKhVgEJxLJYwq5+gLRsb/mo5Xi0vTQ5EYuojH0=; b=TajcPBykzMV0eL9dLEJzwK4tSkjkhUlb+1GJCEKbUfsiz9/1tFOGOSQdyIRkrK6EwR N8uJW6oXfym0FiLE7W3mnioTQMVWIrONt3kBEgH5pCY042GKYJ7hsQ+amXmAAqrqk9pj rbIo3S2z2oiTEUV3pGQiqEsZ6S8v1UYYxZWDa4+xQaBMa4oDODNgMjOAaq7GquZZHTu/ wGUN/f/A3ikRl4r9ZC65b9mH6CAl8kdwhIFT5eLhruReXkT2ta3qfe2cwrcDD5PQZRsn 5+xZFV/vJqfWyPR1/Rx6IOHdJWF6ozeqhqYIpylPpqBoKVenns5kqBTreMataC1zSKpf BDjg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1713771792; x=1714376592; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=zrl4akKhVgEJxLJYwq5+gLRsb/mo5Xi0vTQ5EYuojH0=; b=l6FAy8Moa4dmnTJaeInt5fMZgY950t00VQSjEvMvVgnXoQr0yitS5goK25Iydc5PdW zUYk1bMUWOPL4WBNhQc4+n6BVBQBN1G2m80KQVWAN4Fn9lWp9XUsPbE3c8PYVCuvXcym tixKvUH+knCEBxQxhk9fxXBf58GC9rBN+tBNcvyKmOKeGo9tkLCQebKN0XsRvIASYgI6 brLWXh2Yv8IxWEkT/zE1oYtYtGJD3BYmwrZ5QI0xTIlEzZ15QJUGTn2YTUsvW0+BW/rL AyRBM0x5rpf2eU07OAbIs/IqEvsKhn2TwLs54XVuInkx1+3BqCH7GFg0lMHA3CKCmrCw RYNg== X-Gm-Message-State: AOJu0YzFljzxaPRDY10FHBlzV19HU7sqMvse7IGZ2tRt9s/RoLA9Wnp4 FOZJW1fddnha10qY9D6CVgj0VDA91WwRvWGJIprd9i38IczEUNDzywxFK/eTZWqhzH5AxoW2mw3 a X-Google-Smtp-Source: AGHT+IEc9oEJpvxbLzrOG8tkIAbOFSv29i67Gv0xrVCixugzuwunQk9IqJOdyb3sAb6X7Uc5rZD22A== X-Received: by 2002:a17:902:db07:b0:1e4:bd90:f1d4 with SMTP id m7-20020a170902db0700b001e4bd90f1d4mr11064583plx.40.1713771792141; Mon, 22 Apr 2024 00:43:12 -0700 (PDT) Received: from libai.bytedance.net ([61.213.176.11]) by smtp.gmail.com with ESMTPSA id e11-20020a170902784b00b001e0bae4490fsm7595634pln.154.2024.04.22.00.43.09 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 22 Apr 2024 00:43:11 -0700 (PDT) From: zhenwei pi To: linux-kernel@vger.kernel.org, linux-mm@kvack.org, virtualization@lists.linux.dev Cc: mst@redhat.com, david@redhat.com, jasowang@redhat.com, xuanzhuo@linux.alibaba.com, akpm@linux-foundation.org, zhenwei pi Subject: [PATCH v2 3/4] virtio_balloon: introduce memory allocation stall counter Date: Mon, 22 Apr 2024 15:42:53 +0800 Message-Id: <20240422074254.1440457-4-pizhenwei@bytedance.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240422074254.1440457-1-pizhenwei@bytedance.com> References: <20240422074254.1440457-1-pizhenwei@bytedance.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Memory allocation stall counter represents the performance/latency of memory allocation, expose this counter to the host side by virtio balloon device via out-of-bound way. Signed-off-by: zhenwei pi Acked-by: David Hildenbrand --- drivers/virtio/virtio_balloon.c | 8 ++++++++ include/uapi/linux/virtio_balloon.h | 6 ++++-- 2 files changed, 12 insertions(+), 2 deletions(-) diff --git a/drivers/virtio/virtio_balloon.c b/drivers/virtio/virtio_balloo= n.c index 87a1d6fa77fb..ab039e83bc6f 100644 --- a/drivers/virtio/virtio_balloon.c +++ b/drivers/virtio/virtio_balloon.c @@ -323,6 +323,8 @@ static inline unsigned int update_balloon_vm_stats(stru= ct virtio_balloon *vb, #ifdef CONFIG_VM_EVENT_COUNTERS unsigned long events[NR_VM_EVENT_ITEMS]; unsigned int idx =3D start; + unsigned int zid; + unsigned long stall =3D 0; =20 all_vm_events(events); update_stat(vb, idx++, VIRTIO_BALLOON_S_SWAP_IN, @@ -333,6 +335,12 @@ static inline unsigned int update_balloon_vm_stats(str= uct virtio_balloon *vb, update_stat(vb, idx++, VIRTIO_BALLOON_S_MINFLT, events[PGFAULT]); update_stat(vb, idx++, VIRTIO_BALLOON_S_OOM_KILL, events[OOM_KILL]); =20 + /* sum all the stall events */ + for (zid =3D 0; zid < MAX_NR_ZONES; zid++) + stall +=3D events[ALLOCSTALL_NORMAL - ZONE_NORMAL + zid]; + + update_stat(vb, idx++, VIRTIO_BALLOON_S_ALLOC_STALL, stall); + #ifdef CONFIG_HUGETLB_PAGE update_stat(vb, idx++, VIRTIO_BALLOON_S_HTLB_PGALLOC, events[HTLB_BUDDY_PGALLOC]); diff --git a/include/uapi/linux/virtio_balloon.h b/include/uapi/linux/virti= o_balloon.h index b17bbe033697..487b893a160e 100644 --- a/include/uapi/linux/virtio_balloon.h +++ b/include/uapi/linux/virtio_balloon.h @@ -72,7 +72,8 @@ struct virtio_balloon_config { #define VIRTIO_BALLOON_S_HTLB_PGALLOC 8 /* Hugetlb page allocations */ #define VIRTIO_BALLOON_S_HTLB_PGFAIL 9 /* Hugetlb page allocation failu= res */ #define VIRTIO_BALLOON_S_OOM_KILL 10 /* OOM killer invocations */ -#define VIRTIO_BALLOON_S_NR 11 +#define VIRTIO_BALLOON_S_ALLOC_STALL 11 /* Stall count of memory allocat= oin */ +#define VIRTIO_BALLOON_S_NR 12 =20 #define VIRTIO_BALLOON_S_NAMES_WITH_PREFIX(VIRTIO_BALLOON_S_NAMES_prefix) = { \ VIRTIO_BALLOON_S_NAMES_prefix "swap-in", \ @@ -85,7 +86,8 @@ struct virtio_balloon_config { VIRTIO_BALLOON_S_NAMES_prefix "disk-caches", \ VIRTIO_BALLOON_S_NAMES_prefix "hugetlb-allocations", \ VIRTIO_BALLOON_S_NAMES_prefix "hugetlb-failures", \ - VIRTIO_BALLOON_S_NAMES_prefix "oom-kills" \ + VIRTIO_BALLOON_S_NAMES_prefix "oom-kills", \ + VIRTIO_BALLOON_S_NAMES_prefix "alloc-stalls" \ } =20 #define VIRTIO_BALLOON_S_NAMES VIRTIO_BALLOON_S_NAMES_WITH_PREFIX("") --=20 2.34.1 From nobody Wed Feb 11 04:00:54 2026 Received: from mail-pl1-f171.google.com (mail-pl1-f171.google.com [209.85.214.171]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 8AA7951C4B for ; Mon, 22 Apr 2024 07:43:16 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.171 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1713771797; cv=none; b=O3imbOQ4gHK9opFc/C1bownQu0YxG95dqfaQ/7WrNwQaSnWPTv+9MyT6EKRI9VU3Hwic3NRnoosBy4DLcWRpsUikBml7KvyVGfJA+DQIe2HnLDdAecVF1GJWcBPbuM3h8ZO/5saA4uKzE1OmZXOQrQEe/jAIydhhpUXMUsoeNsk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1713771797; c=relaxed/simple; bh=B7vOTsSVYsGFUvhOXvJRAjHPNDIwccqInso5hXY7xSM=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=tVkTzVHBumubVuybBGh1y61aF1DLQvrQWBFiPG+mP02YTh5tg4IB4sJC/WgKnPPFHKWxZUCVB03W4iKqvgQzIdE8K3Tdkp4pdReCANrzDyFE7vKlf6IzmaGy6CJ89pzVFoP1tNH6IRQH8ABqVhUJSHW0mcgXV2pxdAxZQC5I7bs= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com; spf=pass smtp.mailfrom=bytedance.com; dkim=pass (2048-bit key) header.d=bytedance.com header.i=@bytedance.com header.b=PvONompd; arc=none smtp.client-ip=209.85.214.171 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=bytedance.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=bytedance.com header.i=@bytedance.com header.b="PvONompd" Received: by mail-pl1-f171.google.com with SMTP id d9443c01a7336-1e4bf0b3e06so39389555ad.1 for ; Mon, 22 Apr 2024 00:43:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1713771795; x=1714376595; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=bBFXolLQYa78XCdQIXx8hSsPDNH1h31ohatbB+q7SSA=; b=PvONompda1dfJWMA4hcgQXLuxfy8W0c47dT8l4VjtbvsdZMHh1/H+1G77fxzOB/2o4 sAq4/fuQvArbunkiUxQXvV32zBLQbmjrRuH7UUmfqyUnTHTYVrlM9fBpiAe/BpP3zU3J qTN3jnAFsftUmHVUSvuApgN/8DAeM1F/TtMVsSSWTv9iOq9Q8xQmErbsoKZuWq6YzXKQ ShfUM8yysTxW39Nr8U6cQxG+DqQvvwlb2mE1Gg/WvZEpiHVqWgvBppN6VwCJWzf/5JmT g0kdz4a1U0XPcCvllo+pqq1HYfyukVg04/RKktlngHZ3OSY6jOgG1hr6epIjNr+vGjcH N5vA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1713771795; x=1714376595; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=bBFXolLQYa78XCdQIXx8hSsPDNH1h31ohatbB+q7SSA=; b=SP2fl25YF4Y4uOTOX+e4TYNkTaAEm7pQ3ZBab5pApVxyKVRYNsbhiGTeWGA9QqDxDc 7Sa7SdaX9yHZidJlMGi0PqG/PmikYHS1RMBhqk4lvHTb+3Y7QTR7qtUV5e2WzFvEbBOr pDeboRmH8BAbfNibSENbC9J85F+D26Axh6/eC17rEybFbnoBTHfLYJ+xsF7oyNEg88OG hGAxxyiwx4xkdqU5ag/T/YtSJOZQqzNXJWcyFKvDFl9HaP3S4tmhUePMccySgo+Vzf/4 izcMAXHfA289fxtBS1iIIs7gnYpMneehJ5GUmqP8/ef7uIp5+YxSGjUvQP7KsvXi6Not 58wA== X-Gm-Message-State: AOJu0YzV/+2Pct/9PYMH8uJxKEoo4UjbGZpv9Ah554abgpUtf8jXBXn0 gf6IFn90otHtaNQpNoLOuuP3jeNXsvnaQASVrTzp1dyIT147vhKJwYuSmGuBNlIvS7pFZx+kCzb E X-Google-Smtp-Source: AGHT+IEvHT3XYAnqbJeBRBtYXwHjzJuRWUr3q0IHkM4UWpx7JR4+gVdQqRx7dbo9nSIw0pAVB8jXbw== X-Received: by 2002:a17:902:784f:b0:1e4:d548:818f with SMTP id e15-20020a170902784f00b001e4d548818fmr9126943pln.58.1713771795591; Mon, 22 Apr 2024 00:43:15 -0700 (PDT) Received: from libai.bytedance.net ([61.213.176.11]) by smtp.gmail.com with ESMTPSA id e11-20020a170902784b00b001e0bae4490fsm7595634pln.154.2024.04.22.00.43.12 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 22 Apr 2024 00:43:15 -0700 (PDT) From: zhenwei pi To: linux-kernel@vger.kernel.org, linux-mm@kvack.org, virtualization@lists.linux.dev Cc: mst@redhat.com, david@redhat.com, jasowang@redhat.com, xuanzhuo@linux.alibaba.com, akpm@linux-foundation.org, zhenwei pi Subject: [PATCH v2 4/4] virtio_balloon: introduce memory scan/reclaim info Date: Mon, 22 Apr 2024 15:42:54 +0800 Message-Id: <20240422074254.1440457-5-pizhenwei@bytedance.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240422074254.1440457-1-pizhenwei@bytedance.com> References: <20240422074254.1440457-1-pizhenwei@bytedance.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Expose memory scan/reclaim information to the host side via virtio balloon device. Now we have a metric to analyze the memory performance: y: counter increases n: counter does not changes h: the rate of counter change is high l: the rate of counter change is low OOM: VIRTIO_BALLOON_S_OOM_KILL STALL: VIRTIO_BALLOON_S_ALLOC_STALL ASCAN: VIRTIO_BALLOON_S_SCAN_ASYNC DSCAN: VIRTIO_BALLOON_S_SCAN_DIRECT ARCLM: VIRTIO_BALLOON_S_RECLAIM_ASYNC DRCLM: VIRTIO_BALLOON_S_RECLAIM_DIRECT - OOM[y], STALL[*], ASCAN[*], DSCAN[*], ARCLM[*], DRCLM[*]: the guest runs under really critial memory pressure - OOM[n], STALL[h], ASCAN[*], DSCAN[l], ARCLM[*], DRCLM[l]: the memory allocation stalls due to cgroup, not the global memory pressure. - OOM[n], STALL[h], ASCAN[*], DSCAN[h], ARCLM[*], DRCLM[h]: the memory allocation stalls due to global memory pressure. The performance gets hurt a lot. A high ratio between DRCLM/DSCAN shows quite effective memory reclaiming. - OOM[n], STALL[h], ASCAN[*], DSCAN[h], ARCLM[*], DRCLM[l]: the memory allocation stalls due to global memory pressure. the ratio between DRCLM/DSCAN gets low, the guest OS is thrashing heavily, the serious case leads poor performance and difficult trouble shooting. Ex, sshd may block on memory allocation when accepting new connections, a user can't login a VM by ssh command. - OOM[n], STALL[n], ASCAN[h], DSCAN[n], ARCLM[l], DRCLM[n]: the low ratio between ARCLM/ASCAN shows that the guest tries to reclaim more memory, but it can't. Once more memory is required in future, it will struggle to reclaim memory. Acked-by: David Hildenbrand Signed-off-by: zhenwei pi --- drivers/virtio/virtio_balloon.c | 9 +++++++++ include/uapi/linux/virtio_balloon.h | 12 ++++++++++-- 2 files changed, 19 insertions(+), 2 deletions(-) diff --git a/drivers/virtio/virtio_balloon.c b/drivers/virtio/virtio_balloo= n.c index ab039e83bc6f..e45146fde164 100644 --- a/drivers/virtio/virtio_balloon.c +++ b/drivers/virtio/virtio_balloon.c @@ -341,6 +341,15 @@ static inline unsigned int update_balloon_vm_stats(str= uct virtio_balloon *vb, =20 update_stat(vb, idx++, VIRTIO_BALLOON_S_ALLOC_STALL, stall); =20 + update_stat(vb, idx++, VIRTIO_BALLOON_S_ASYNC_SCAN, + pages_to_bytes(events[PGSCAN_KSWAPD])); + update_stat(vb, idx++, VIRTIO_BALLOON_S_DIRECT_SCAN, + pages_to_bytes(events[PGSCAN_DIRECT])); + update_stat(vb, idx++, VIRTIO_BALLOON_S_ASYNC_RECLAIM, + pages_to_bytes(events[PGSTEAL_KSWAPD])); + update_stat(vb, idx++, VIRTIO_BALLOON_S_DIRECT_RECLAIM, + pages_to_bytes(events[PGSTEAL_DIRECT])); + #ifdef CONFIG_HUGETLB_PAGE update_stat(vb, idx++, VIRTIO_BALLOON_S_HTLB_PGALLOC, events[HTLB_BUDDY_PGALLOC]); diff --git a/include/uapi/linux/virtio_balloon.h b/include/uapi/linux/virti= o_balloon.h index 487b893a160e..ee35a372805d 100644 --- a/include/uapi/linux/virtio_balloon.h +++ b/include/uapi/linux/virtio_balloon.h @@ -73,7 +73,11 @@ struct virtio_balloon_config { #define VIRTIO_BALLOON_S_HTLB_PGFAIL 9 /* Hugetlb page allocation failu= res */ #define VIRTIO_BALLOON_S_OOM_KILL 10 /* OOM killer invocations */ #define VIRTIO_BALLOON_S_ALLOC_STALL 11 /* Stall count of memory allocat= oin */ -#define VIRTIO_BALLOON_S_NR 12 +#define VIRTIO_BALLOON_S_ASYNC_SCAN 12 /* Amount of memory scanned asyn= chronously */ +#define VIRTIO_BALLOON_S_DIRECT_SCAN 13 /* Amount of memory scanned dire= ctly */ +#define VIRTIO_BALLOON_S_ASYNC_RECLAIM 14 /* Amount of memory reclaimed as= ynchronously */ +#define VIRTIO_BALLOON_S_DIRECT_RECLAIM 15 /* Amount of memory reclaimed d= irectly */ +#define VIRTIO_BALLOON_S_NR 16 =20 #define VIRTIO_BALLOON_S_NAMES_WITH_PREFIX(VIRTIO_BALLOON_S_NAMES_prefix) = { \ VIRTIO_BALLOON_S_NAMES_prefix "swap-in", \ @@ -87,7 +91,11 @@ struct virtio_balloon_config { VIRTIO_BALLOON_S_NAMES_prefix "hugetlb-allocations", \ VIRTIO_BALLOON_S_NAMES_prefix "hugetlb-failures", \ VIRTIO_BALLOON_S_NAMES_prefix "oom-kills", \ - VIRTIO_BALLOON_S_NAMES_prefix "alloc-stalls" \ + VIRTIO_BALLOON_S_NAMES_prefix "alloc-stalls", \ + VIRTIO_BALLOON_S_NAMES_prefix "async-scans", \ + VIRTIO_BALLOON_S_NAMES_prefix "direct-scans", \ + VIRTIO_BALLOON_S_NAMES_prefix "async-reclaims", \ + VIRTIO_BALLOON_S_NAMES_prefix "direct-reclaims" \ } =20 #define VIRTIO_BALLOON_S_NAMES VIRTIO_BALLOON_S_NAMES_WITH_PREFIX("") --=20 2.34.1