From nobody Mon Feb 9 13:38:14 2026 Received: from mail-il1-f175.google.com (mail-il1-f175.google.com [209.85.166.175]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id DDD2A39AC9 for ; Mon, 15 Apr 2024 08:41:31 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.166.175 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1713170494; cv=none; b=HTAiHgPoX0U5uuRdNWXQhTwf1UrJcmFL7Nx0BnEKdbc6sEpjGB1hwhTJx6nGq+jFXgWGp0l0OCwmrNdVeNXJYuk8opTgtD3UvBCCmE/wAHPlG7NZeh/NgJUpe5yumTFnI2hdKHzwV3Sz5zYchJYFnatNGEebzjEkU+h67SbJ4/I= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1713170494; c=relaxed/simple; bh=7/Cb3EbaM1UDAA5Kb72FztigJLQVxjyacKvxcUVA7Ks=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=PEqfT/DhuD9lN/IwaPK9UcEgZmeDRpQ7IVK4gCoTXqPX7DoSXcQxYqKHE9KNWHplwUqN7xuO1yXET6AlD9v7V+ixfgI2D2+IhS761vkT7i1uH/5TGpg6XOoTQge0SBoIIVB0e1e7VRUyLx+StyfxzvY/syC1/zIOjfw4CgwZcF4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com; spf=pass smtp.mailfrom=bytedance.com; dkim=pass (2048-bit key) header.d=bytedance.com header.i=@bytedance.com header.b=UWvTNBwq; arc=none smtp.client-ip=209.85.166.175 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=bytedance.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=bytedance.com header.i=@bytedance.com header.b="UWvTNBwq" Received: by mail-il1-f175.google.com with SMTP id e9e14a558f8ab-36a38c13550so15206985ab.0 for ; Mon, 15 Apr 2024 01:41:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1713170491; x=1713775291; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=12B6L10PsA9Jh230IG9uEJXqT+JXXSEjbmkq482lFPs=; b=UWvTNBwqEb30LmbkpjtvpeM+m7Wza2dUY909RQ9/cP8cXG879IXd1UUcYM6HI7QtBp 6MtFOura1H6LLbuXDiyTFiwIilD8qBoFkgQrvMHVVP3J6PKgQuezISQi+kKtirNDTePR CS7k0CzY9SJH+fQbayXr4dMarh38Lmsi8zBbEKv6tPQnBdbX+ySLgrEQyjNb8tMi8e8Q oX2F84MdRJNZFz1Oz422jPfqmcTuofy9ZBePBnGSVhYmAXxRUyAXnARj/uv/U7FAobrX yZSfOBUU2EUEeD3l9+/pjVSg8Ha1q8HIH8Vpt/mAE88yChuleqS5vzD8PxGZmiedVZBe AxZg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1713170491; x=1713775291; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=12B6L10PsA9Jh230IG9uEJXqT+JXXSEjbmkq482lFPs=; b=EPCubo31DU9p2jwoQ5KfnI1gjyCO++Y0YeyutS7cgzyE+NXekZO/hDDpHxUaKUsitU yu7SsxpaBr3WfeycntLeGxSLqeSo0UL3mTgZeohkVq7E3mLQvKC4ERDcP2//XN3pFDkK CYFEhnrxK3t/QWH52gkn1fYhTOQ371ii7NlqXQWzz1P/AYiVsISM3AdrBK1DG0fVtiMJ irbVh+XFMdCsnS9c8oe73/GvokrJTpphl2v2bT4FdRvIW7esrfUwvwoToFOCY9sItwY/ a9sb5YH+6YPLLbMgA7h3vsn3hvFfVSJDuSHWxltT8xw/BhDcx0cXkyDEV3mXh3DdBX58 hANA== X-Gm-Message-State: AOJu0Yy+j8NKuqXGYdVyMqLgAbsCNh3ziw2gSHyynYOyLZZxOlhazVoM g21Ab8sJjLEvurr6u91h1VkC1Dkh4s/7OAw2mwqnsS8bjRlPhErROkqsYLZimtfsZ9xB+EDIUBW T X-Google-Smtp-Source: AGHT+IHu3Id3boHE0EmOQ8pWwfFovlaejeViuBMnTD/yfIQaRW2940ghQ6qCZwrhul6FRmwhdLfOtQ== X-Received: by 2002:a92:cd81:0:b0:36b:214:bb21 with SMTP id r1-20020a92cd81000000b0036b0214bb21mr11364794ilb.25.1713170490833; Mon, 15 Apr 2024 01:41:30 -0700 (PDT) Received: from libai.bytedance.net ([61.213.176.11]) by smtp.gmail.com with ESMTPSA id k187-20020a636fc4000000b005d6a0b2efb3sm6575685pgc.21.2024.04.15.01.41.27 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 15 Apr 2024 01:41:30 -0700 (PDT) From: zhenwei pi To: linux-kernel@vger.kernel.org, linux-mm@kvack.org, virtualization@lists.linux.dev Cc: mst@redhat.com, david@redhat.com, jasowang@redhat.com, xuanzhuo@linux.alibaba.com, akpm@linux-foundation.org, zhenwei pi Subject: [RFC 1/3] virtio_balloon: introduce oom-kill invocations Date: Mon, 15 Apr 2024 16:41:11 +0800 Message-Id: <20240415084113.1203428-2-pizhenwei@bytedance.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240415084113.1203428-1-pizhenwei@bytedance.com> References: <20240415084113.1203428-1-pizhenwei@bytedance.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" When the guest OS runs under critical memory pressure, the guest starts to kill processes. A guest monitor agent may scan 'oom_kill' from /proc/vmstat, and reports the OOM KILL event. However, the agent may be killed and we will loss this critical event(and the later events). For now we can also grep for magic words in guest kernel log from host side. Rather than this unstable way, virtio balloon reports OOM-KILL invocations instead. Signed-off-by: zhenwei pi --- drivers/virtio/virtio_balloon.c | 2 ++ include/uapi/linux/virtio_balloon.h | 6 ++++-- 2 files changed, 6 insertions(+), 2 deletions(-) diff --git a/drivers/virtio/virtio_balloon.c b/drivers/virtio/virtio_balloo= n.c index 1f5b3dd31fcf..fd8daa742734 100644 --- a/drivers/virtio/virtio_balloon.c +++ b/drivers/virtio/virtio_balloon.c @@ -352,6 +352,8 @@ static unsigned int update_balloon_stats(struct virtio_= balloon *vb) pages_to_bytes(available)); update_stat(vb, idx++, VIRTIO_BALLOON_S_CACHES, pages_to_bytes(caches)); + update_stat(vb, idx++, VIRTIO_BALLOON_S_OOM_KILL, + events[OOM_KILL]); =20 return idx; } diff --git a/include/uapi/linux/virtio_balloon.h b/include/uapi/linux/virti= o_balloon.h index ddaa45e723c4..cde5547e64a7 100644 --- a/include/uapi/linux/virtio_balloon.h +++ b/include/uapi/linux/virtio_balloon.h @@ -71,7 +71,8 @@ struct virtio_balloon_config { #define VIRTIO_BALLOON_S_CACHES 7 /* Disk caches */ #define VIRTIO_BALLOON_S_HTLB_PGALLOC 8 /* Hugetlb page allocations */ #define VIRTIO_BALLOON_S_HTLB_PGFAIL 9 /* Hugetlb page allocation failu= res */ -#define VIRTIO_BALLOON_S_NR 10 +#define VIRTIO_BALLOON_S_OOM_KILL 10 /* OOM killer invocations */ +#define VIRTIO_BALLOON_S_NR 11 =20 #define VIRTIO_BALLOON_S_NAMES_WITH_PREFIX(VIRTIO_BALLOON_S_NAMES_prefix) = { \ VIRTIO_BALLOON_S_NAMES_prefix "swap-in", \ @@ -83,7 +84,8 @@ struct virtio_balloon_config { VIRTIO_BALLOON_S_NAMES_prefix "available-memory", \ VIRTIO_BALLOON_S_NAMES_prefix "disk-caches", \ VIRTIO_BALLOON_S_NAMES_prefix "hugetlb-allocations", \ - VIRTIO_BALLOON_S_NAMES_prefix "hugetlb-failures" \ + VIRTIO_BALLOON_S_NAMES_prefix "hugetlb-failures", \ + VIRTIO_BALLOON_S_NAMES_prefix "oom-kill" \ } =20 #define VIRTIO_BALLOON_S_NAMES VIRTIO_BALLOON_S_NAMES_WITH_PREFIX("") --=20 2.34.1 From nobody Mon Feb 9 13:38:14 2026 Received: from mail-oo1-f45.google.com (mail-oo1-f45.google.com [209.85.161.45]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 4294F3BBDC for ; Mon, 15 Apr 2024 08:41:35 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.161.45 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1713170496; cv=none; b=kovKBfCHwVZuYUsN4/VkTOEAswYu0E4v9ZCVBiOrnGQ/868EMr3XGiWfjQdwiULPVWhqj2IGG4MqUghT3QBC1Z+eQbnsiRjUewBeIIbeYUiYOcO0KknuNKcHDM/odznYOrzdEM7l4PUPxUOxJIh9l129Wm6N9dSl6Cl+pnZIe+I= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1713170496; c=relaxed/simple; bh=vZqUoc3gYd+I/LHSXlUpLfccZsnhlmnxJYwDUfMYSzk=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=o8JEGKGYMpXmw8LGxYdwpEhZGlFjU8lqqvz8xte1d07vzyZoG4GAFqhiCd0YoYL70nqrKuF9ptgNIwqgK7DJlKmA5vEXhx9b4KLrDnXnA14Tz75g/fcRduXESyaOhAgBD62LMa7lU+mCSKIsTVMCIZUPlyjPURBcKGhUsJZjuE0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com; spf=pass smtp.mailfrom=bytedance.com; dkim=pass (2048-bit key) header.d=bytedance.com header.i=@bytedance.com header.b=FEMyBWQV; arc=none smtp.client-ip=209.85.161.45 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=bytedance.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=bytedance.com header.i=@bytedance.com header.b="FEMyBWQV" Received: by mail-oo1-f45.google.com with SMTP id 006d021491bc7-5aa3af24775so2267153eaf.0 for ; Mon, 15 Apr 2024 01:41:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1713170494; x=1713775294; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=gn/fmyyLOBGP9D7Ah7/te94Cqx4P0YQuEb1tR3Gfsk4=; b=FEMyBWQV2FrEQgQLSm8cNDYj8TnCQPIUWlJjso6vLNXBGGzGA8CueFFl6sE0DdsLLH rSoK1RFxMX9b6f5YWfVWREGxq3hRBwqgC1OR69ZOPhTd8fP9re6lkNCqrULsjWX+4h3a xMyJM1GCOBgKGmriMUgDcgyuujV9VfnEznnBA1Y2H8eTDIxtkevpsgZ82y0ZNaD9Wb+l AXWFs4SWAz9cnnuh/vsbIuePzDCHroX7+R4udDcSxpP8nTh6Bxwqry7lROUvdTCFr+Iw 6KTmDyO+qowVOZIez5+jyuDsmm7iKeVQVt4FeJWHHX7FKF/aEEVNLsqVgzLbXacPmhtg pePg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1713170494; x=1713775294; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=gn/fmyyLOBGP9D7Ah7/te94Cqx4P0YQuEb1tR3Gfsk4=; b=mvxcBhoL5GgOQnZSoxVvk5EjWnUpk9Rud3Vjx94W3iESSokseVbLaAckCNncrjJTEJ 9v9ao6ISsWyOxDL7WAcc8T7UuNaxuh8KFpnqWYSWYSJK2tyGB/J2NPVgaMru9F3jhNeM sDLtpxUmGelKGKnGK5YHRzWtf9iLJmwAbcam+i6k9W19sUMusp7EdDGxHbfgXegp2hAV C6XYa2afOqTXVI5eH+ZEKiT//WP/FkcntZrXARoMKtde4yfYzuIJYnTiPKl739/nEd7O iEpVQypIycfoMBlNNjubt01fu1qRmyP8hlAyw0/YV/VdywAGTX8/vQEh+PeUHDpiHcrD KdLw== X-Gm-Message-State: AOJu0YzE5Btb/2i1V/L8bXTwjblMhhe7iSlHZhsgPN38IcOpHCFD8KPA L/15XIxTO8R2f3ujhijiKizXRnos1DG5suht81U9jPE5SDt4GzFdWaXmRBxuAYXEVL3qkfIbk9I B X-Google-Smtp-Source: AGHT+IETdp6r2QsgZsbcC8cYUyAoKu02zRWiFTX/N1oYt3Eg7GPbdngrRxjNkXqYCiCkcAyLjTOyFg== X-Received: by 2002:a05:6358:1396:b0:183:6427:10b6 with SMTP id n22-20020a056358139600b00183642710b6mr10637354rwi.19.1713170494297; Mon, 15 Apr 2024 01:41:34 -0700 (PDT) Received: from libai.bytedance.net ([61.213.176.11]) by smtp.gmail.com with ESMTPSA id k187-20020a636fc4000000b005d6a0b2efb3sm6575685pgc.21.2024.04.15.01.41.31 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 15 Apr 2024 01:41:33 -0700 (PDT) From: zhenwei pi To: linux-kernel@vger.kernel.org, linux-mm@kvack.org, virtualization@lists.linux.dev Cc: mst@redhat.com, david@redhat.com, jasowang@redhat.com, xuanzhuo@linux.alibaba.com, akpm@linux-foundation.org, zhenwei pi Subject: [RFC 2/3] virtio_balloon: introduce memory allocation stall counter Date: Mon, 15 Apr 2024 16:41:12 +0800 Message-Id: <20240415084113.1203428-3-pizhenwei@bytedance.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240415084113.1203428-1-pizhenwei@bytedance.com> References: <20240415084113.1203428-1-pizhenwei@bytedance.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Memory allocation stall counter represents the performance/latency of memory allocation, expose this counter to the host side by virtio balloon device via out-of-bound way. Signed-off-by: zhenwei pi --- drivers/virtio/virtio_balloon.c | 19 ++++++++++++++++++- include/uapi/linux/virtio_balloon.h | 6 ++++-- 2 files changed, 22 insertions(+), 3 deletions(-) diff --git a/drivers/virtio/virtio_balloon.c b/drivers/virtio/virtio_balloo= n.c index fd8daa742734..4b9c9569f6e5 100644 --- a/drivers/virtio/virtio_balloon.c +++ b/drivers/virtio/virtio_balloon.c @@ -321,7 +321,7 @@ static unsigned int update_balloon_stats(struct virtio_= balloon *vb) unsigned long events[NR_VM_EVENT_ITEMS]; struct sysinfo i; unsigned int idx =3D 0; - long available; + long available, stall =3D 0; unsigned long caches; =20 all_vm_events(events); @@ -355,6 +355,23 @@ static unsigned int update_balloon_stats(struct virtio= _balloon *vb) update_stat(vb, idx++, VIRTIO_BALLOON_S_OOM_KILL, events[OOM_KILL]); =20 + /* sum all the stall event */ +#ifdef CONFIG_ZONE_DMA + stall +=3D events[ALLOCSTALL_DMA]; +#endif +#ifdef CONFIG_ZONE_DMA32 + stall +=3D events[ALLOCSTALL_DMA32]; +#endif +#ifdef CONFIG_HIGHMEM + stall +=3D events[ALLOCSTALL_HIGH]; +#endif +#ifdef CONFIG_ZONE_DEVICE + stall +=3D events[ALLOCSTALL_DEVICE]; +#endif + stall +=3D events[ALLOCSTALL_NORMAL]; + stall +=3D events[ALLOCSTALL_MOVABLE]; + update_stat(vb, idx++, VIRTIO_BALLOON_S_ALLOC_STALL, stall); + return idx; } =20 diff --git a/include/uapi/linux/virtio_balloon.h b/include/uapi/linux/virti= o_balloon.h index cde5547e64a7..13d0c32ba27c 100644 --- a/include/uapi/linux/virtio_balloon.h +++ b/include/uapi/linux/virtio_balloon.h @@ -72,7 +72,8 @@ struct virtio_balloon_config { #define VIRTIO_BALLOON_S_HTLB_PGALLOC 8 /* Hugetlb page allocations */ #define VIRTIO_BALLOON_S_HTLB_PGFAIL 9 /* Hugetlb page allocation failu= res */ #define VIRTIO_BALLOON_S_OOM_KILL 10 /* OOM killer invocations */ -#define VIRTIO_BALLOON_S_NR 11 +#define VIRTIO_BALLOON_S_ALLOC_STALL 11 /* Stall count of memory allocat= oin */ +#define VIRTIO_BALLOON_S_NR 12 =20 #define VIRTIO_BALLOON_S_NAMES_WITH_PREFIX(VIRTIO_BALLOON_S_NAMES_prefix) = { \ VIRTIO_BALLOON_S_NAMES_prefix "swap-in", \ @@ -85,7 +86,8 @@ struct virtio_balloon_config { VIRTIO_BALLOON_S_NAMES_prefix "disk-caches", \ VIRTIO_BALLOON_S_NAMES_prefix "hugetlb-allocations", \ VIRTIO_BALLOON_S_NAMES_prefix "hugetlb-failures", \ - VIRTIO_BALLOON_S_NAMES_prefix "oom-kill" \ + VIRTIO_BALLOON_S_NAMES_prefix "oom-kill", \ + VIRTIO_BALLOON_S_NAMES_prefix "alloc-stall" \ } =20 #define VIRTIO_BALLOON_S_NAMES VIRTIO_BALLOON_S_NAMES_WITH_PREFIX("") --=20 2.34.1 From nobody Mon Feb 9 13:38:14 2026 Received: from mail-oa1-f46.google.com (mail-oa1-f46.google.com [209.85.160.46]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C304A383A3 for ; Mon, 15 Apr 2024 08:41:38 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.160.46 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1713170500; cv=none; b=ip1RgmUIrQ7pxjjoQsLeFRlAIVh3PmXVmT072hpHpqRZTXtxHxQ4bfpYQE8X/w0S9zvboQDVtV18hn0tk5R5Xi4OtxOpnzuAAFvKfsdzGLZ338nY+g9BtQYrW4L10LVAIzazUTDTLi2db07IGTb47TGbzyvlCq7cAF+RE5MjNX4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1713170500; c=relaxed/simple; bh=NFrR2exu6d6NexTbth3nKYMeBa96W+PF0xe/+85rGTE=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=r4C/HOkGXgGaSBA/YF2h6HXF6m7U4PbUmDmoGo1OYywsB398SAxB8nKIOTjOwNDCGqABmeZ2DpmvgsLsiDr1r/uZ0AhTTlP7ZsiMvvGU58NmJpDGH+Ow9FHqQMYxtH8Xa/JFpzfyrs+qtPiQ6CfxksO6Vto8oJUH1lyh9JQDU64= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com; spf=pass smtp.mailfrom=bytedance.com; dkim=pass (2048-bit key) header.d=bytedance.com header.i=@bytedance.com header.b=hwbUmXLu; arc=none smtp.client-ip=209.85.160.46 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=bytedance.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=bytedance.com header.i=@bytedance.com header.b="hwbUmXLu" Received: by mail-oa1-f46.google.com with SMTP id 586e51a60fabf-22efc6b8dc5so2148666fac.0 for ; Mon, 15 Apr 2024 01:41:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1713170497; x=1713775297; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=jVSyUCsRBJHReEwHJLY9AUjYGa31t9M8IXFXE6DP2gA=; b=hwbUmXLuxKWfRKiLajsyQk/42oJOZ3pFvBzHhnQrGgREV5//YBvTikDu3JTmS+2o58 61Sd+s5DQNlNTIVpBzwhTzPDKaY4QWJ6Rj5ZtRxndLyQFMZj+zJcJ+rowQHY3VTkyK2Y P9IA97lHYPJeE+fj3Evk9o7QvwwjubrRlIV+ZA8SsiNVUGlHLTg0ITBVgIl5PP7RLjhO ULVUYp3dXhyZErE5P1f6Z/PEEIxAlMnwPyfAgzKWbtGJ6/FSyjxfVBEiG4I8BMFZI7QB aj+CgJaU+2sCw47xasqjx5RWBHEwsSMK94u5a650QdcPIUhXMQ4AvcyBEJH3c9VajH5S 2wQw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1713170497; x=1713775297; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=jVSyUCsRBJHReEwHJLY9AUjYGa31t9M8IXFXE6DP2gA=; b=iZND8gVHbu2XJyt3wPJVmvl+LibhWNwdWJU/xde7ahsMYhQap3+cyX8I5Yg/C2//zX 0b4BSqwZ4zQx9GpE4kWkbMk7Vob6FeRCXo3Y0HBd+dDSaabk77BDHp2fnDWhbodltTVX VzHhPpYBmCLGs6BcgNkb0UY0J3yxzCV4TLNdBh/OpwXPgtQVt89hdEC08z8oPy1CnHqS JnAuT4F1eVsZj4+/ANf/UPP+ItuP1xNzF349ZOF3ZbsaPVhEubHO2BHVg7DDkQYK70HU lvN6kFJfCvTGhf0LI/PXKU87435lPtXnZxd4kA4nSgOSZMbws0inegQHswJ8Irs+usv/ ujzA== X-Gm-Message-State: AOJu0Yzi4dVS27eNON7rST6MWlx9Yxllwk+OrrjarBoCzKEBTzGyUtul 83IJy17MX2ind5sltKevg4GhM65oAJzj9SQE60AnVZicjXpQP8eCDyx53CiQtx+xl/r7eZoPHdj q X-Google-Smtp-Source: AGHT+IFLEEPCt9VyzH/L1QCrMmvcBH5qldAmYAh+iqUObe92I/ilLOhZ5GsT6F60/+szU0KPLSFuSA== X-Received: by 2002:a05:6870:9113:b0:22f:7513:f20a with SMTP id o19-20020a056870911300b0022f7513f20amr11633580oae.55.1713170497684; Mon, 15 Apr 2024 01:41:37 -0700 (PDT) Received: from libai.bytedance.net ([61.213.176.11]) by smtp.gmail.com with ESMTPSA id k187-20020a636fc4000000b005d6a0b2efb3sm6575685pgc.21.2024.04.15.01.41.34 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 15 Apr 2024 01:41:37 -0700 (PDT) From: zhenwei pi To: linux-kernel@vger.kernel.org, linux-mm@kvack.org, virtualization@lists.linux.dev Cc: mst@redhat.com, david@redhat.com, jasowang@redhat.com, xuanzhuo@linux.alibaba.com, akpm@linux-foundation.org, zhenwei pi Subject: [RFC 3/3] virtio_balloon: introduce memory scan/reclaim info Date: Mon, 15 Apr 2024 16:41:13 +0800 Message-Id: <20240415084113.1203428-4-pizhenwei@bytedance.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240415084113.1203428-1-pizhenwei@bytedance.com> References: <20240415084113.1203428-1-pizhenwei@bytedance.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Expose memory scan/reclaim information to the host side via virtio balloon device. Now we have a metric to analyze the memory performance: y: counter increases n: counter does not changes h: the rate of counter change is high l: the rate of counter change is low OOM: VIRTIO_BALLOON_S_OOM_KILL STALL: VIRTIO_BALLOON_S_ALLOC_STALL ASCAN: VIRTIO_BALLOON_S_SCAN_ASYNC DSCAN: VIRTIO_BALLOON_S_SCAN_DIRECT ARCLM: VIRTIO_BALLOON_S_RECLAIM_ASYNC DRCLM: VIRTIO_BALLOON_S_RECLAIM_DIRECT - OOM[y], STALL[*], ASCAN[*], DSCAN[*], ARCLM[*], DRCLM[*]: the guest runs under really critial memory pressure - OOM[n], STALL[h], ASCAN[*], DSCAN[l], ARCLM[*], DRCLM[l]: the memory allocation stalls due to cgroup, not the global memory pressure. - OOM[n], STALL[h], ASCAN[*], DSCAN[h], ARCLM[*], DRCLM[h]: the memory allocation stalls due to global memory pressure. The performance gets hurt a lot. A high ratio between DRCLM/DSCAN shows quite effective memory reclaiming. - OOM[n], STALL[h], ASCAN[*], DSCAN[h], ARCLM[*], DRCLM[l]: the memory allocation stalls due to global memory pressure. the ratio between DRCLM/DSCAN gets low, the guest OS is thrashing heavily, the serious case leads poor performance and difficult trouble shooting. Ex, sshd may block on memory allocation when accepting new connections, a user can't login a VM by ssh command. - OOM[n], STALL[n], ASCAN[h], DSCAN[n], ARCLM[l], DRCLM[n]: the low ratio between ARCLM/ASCAN shows that the guest tries to reclaim more memory, but it can't. Once more memory is required in future, it will struggle to reclaim memory. Signed-off-by: zhenwei pi --- drivers/virtio/virtio_balloon.c | 9 +++++++++ include/uapi/linux/virtio_balloon.h | 12 ++++++++++-- 2 files changed, 19 insertions(+), 2 deletions(-) diff --git a/drivers/virtio/virtio_balloon.c b/drivers/virtio/virtio_balloo= n.c index 4b9c9569f6e5..7b86514e99d4 100644 --- a/drivers/virtio/virtio_balloon.c +++ b/drivers/virtio/virtio_balloon.c @@ -372,6 +372,15 @@ static unsigned int update_balloon_stats(struct virtio= _balloon *vb) stall +=3D events[ALLOCSTALL_MOVABLE]; update_stat(vb, idx++, VIRTIO_BALLOON_S_ALLOC_STALL, stall); =20 + update_stat(vb, idx++, VIRTIO_BALLOON_S_SCAN_ASYNC, + pages_to_bytes(events[PGSCAN_KSWAPD])); + update_stat(vb, idx++, VIRTIO_BALLOON_S_SCAN_DIRECT, + pages_to_bytes(events[PGSCAN_DIRECT])); + update_stat(vb, idx++, VIRTIO_BALLOON_S_RECLAIM_ASYNC, + pages_to_bytes(events[PGSTEAL_KSWAPD])); + update_stat(vb, idx++, VIRTIO_BALLOON_S_RECLAIM_DIRECT, + pages_to_bytes(events[PGSTEAL_DIRECT])); + return idx; } =20 diff --git a/include/uapi/linux/virtio_balloon.h b/include/uapi/linux/virti= o_balloon.h index 13d0c32ba27c..0875a9cccb01 100644 --- a/include/uapi/linux/virtio_balloon.h +++ b/include/uapi/linux/virtio_balloon.h @@ -73,7 +73,11 @@ struct virtio_balloon_config { #define VIRTIO_BALLOON_S_HTLB_PGFAIL 9 /* Hugetlb page allocation failu= res */ #define VIRTIO_BALLOON_S_OOM_KILL 10 /* OOM killer invocations */ #define VIRTIO_BALLOON_S_ALLOC_STALL 11 /* Stall count of memory allocat= oin */ -#define VIRTIO_BALLOON_S_NR 12 +#define VIRTIO_BALLOON_S_SCAN_ASYNC 12 /* Amount of memory scanned asyn= chronously */ +#define VIRTIO_BALLOON_S_SCAN_DIRECT 13 /* Amount of memory scanned dire= ctly */ +#define VIRTIO_BALLOON_S_RECLAIM_ASYNC 14 /* Amount of memory reclaimed as= ynchronously */ +#define VIRTIO_BALLOON_S_RECLAIM_DIRECT 15 /* Amount of memory reclaimed d= irectly */ +#define VIRTIO_BALLOON_S_NR 16 =20 #define VIRTIO_BALLOON_S_NAMES_WITH_PREFIX(VIRTIO_BALLOON_S_NAMES_prefix) = { \ VIRTIO_BALLOON_S_NAMES_prefix "swap-in", \ @@ -87,7 +91,11 @@ struct virtio_balloon_config { VIRTIO_BALLOON_S_NAMES_prefix "hugetlb-allocations", \ VIRTIO_BALLOON_S_NAMES_prefix "hugetlb-failures", \ VIRTIO_BALLOON_S_NAMES_prefix "oom-kill", \ - VIRTIO_BALLOON_S_NAMES_prefix "alloc-stall" \ + VIRTIO_BALLOON_S_NAMES_prefix "alloc-stall", \ + VIRTIO_BALLOON_S_NAMES_prefix "scan-async", \ + VIRTIO_BALLOON_S_NAMES_prefix "scan-direct", \ + VIRTIO_BALLOON_S_NAMES_prefix "reclaim-async", \ + VIRTIO_BALLOON_S_NAMES_prefix "reclaim-direct" \ } =20 #define VIRTIO_BALLOON_S_NAMES VIRTIO_BALLOON_S_NAMES_WITH_PREFIX("") --=20 2.34.1