From nobody Sat Feb 7 21:11:05 2026 Received: from mail-wm1-f49.google.com (mail-wm1-f49.google.com [209.85.128.49]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 505A938B9AE for ; Mon, 19 Jan 2026 15:44:03 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.49 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1768837444; cv=none; b=VoEiJJmDkHkJKAWMb+dv/PCnH+jA7f/W7Yjaz99tmy2HUjesM80200LTDp7LNzSnlkFw2trbFjpVG5mBt9D2k7xlFFJF8Fjfj4DOvK15LvtG1szHBLqCbTV+6ly7VIt/u+Us5wYHq+09gOVCtECElnpQ7o/DWBs1azctV3HE77g= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1768837444; c=relaxed/simple; bh=2HuE6tbSpLQ62H6pjoqaeKbW/rufO55isJK3aHLjh8U=; h=From:To:Cc:Subject:Date:Message-ID:MIME-Version; b=kKYjFyzP2tEV+WtpzqwAgEvjudFnHox/9N/kxQ0qIWGJXqW3vhzO+OAADl6RG0Smzd5OAaf8UnueLVW27nTGejT0gJSHNhDHiiWUgQ+Qlt9QtOm1SzlQ/lBEsf4lMy0j7jFbroG92uleE6KrRaOBxNdTGfs3Md9UrKjc+sq9D4c= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=JS44HMHU; arc=none smtp.client-ip=209.85.128.49 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="JS44HMHU" Received: by mail-wm1-f49.google.com with SMTP id 5b1f17b1804b1-47ee937ecf2so31865035e9.0 for ; Mon, 19 Jan 2026 07:44:03 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1768837442; x=1769442242; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:from:to:cc:subject:date:message-id:reply-to; bh=EWJKJQsunGDWGcktrSRSGAJYI4Y83xT0WkUFHjnbal0=; b=JS44HMHUypQnedAh1HR0vaQ2GZRCQ+JmfuWEDW2W9+Pm/vFf5RGOicdDdtoTPw56kB QySWhTT4THI4/yGxMYmZ0ZgZAiVrZTakkFpIv3wk3GNOabky9jPFBE5LLYh6k0rqg9GI QxJAbEuPZKctRETlK4A+hZZ36buOBHtYkz0I6rJ9McK2HwLKxhjLN5k3KwEMTpbljjYL NhefBi2SrCRN1b1vP/3vLNvAmHfeSu9W4sZ3i/DsKN9dUvrcWS1SnLejSy5XY0RJj5jU yJyNWKsfdop+dqdQvSatytUoA6Jgz5NA97QxiTvqgmKK4qMeVCASzWx5yOLu4/pFDqp8 V/xA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1768837442; x=1769442242; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:x-gm-gg:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=EWJKJQsunGDWGcktrSRSGAJYI4Y83xT0WkUFHjnbal0=; b=VC65WhYgit1ndLmcFL1RimD8ournN8ysURmAUCsxQhzTmaCclQ3h5QWnvdzJVfWoET 0E9KiSPCRARepN5tGOhfZ/A77yLgCPVKmfwhzz1y/UISNMgR544zdfSVcDccCjpAuKgH /v6T9lLePxcMqA791cXBkwItquWQJ1QS2mYi4xd3cBkD3JC0sUbc7zOmu40g17yHWuOq k8FvkNrrvYs7SWY2dl4aU160X27x5msM6v6UYwRK78ABlo5iTA4tPr0J85Jo8OqFUWXa M1nTdVE1Exaq8X7Gf5Q30qhgmMJKvlXr9Bpj/gzxYlq1/ZuJBpQEPTwqz+bk9N4BRX0w e9Yw== X-Forwarded-Encrypted: i=1; AJvYcCXXapUYcQxp8g810AxbEzxr3zyhy8B7H/K5DwxjaoRrqBhZCxSrSKYqQBHrU4zS16pXwqPHa2WYT+BGeGE=@vger.kernel.org X-Gm-Message-State: AOJu0YxFm7XjDaZm/L+ocYtrHCDAspNl0iZ1ypZe2sfuY/pEETV/XCzn VJMmiLimxe7QqD9E4Fuv5GhzUTWHYkuIUAI9J4viDgKNlBq7ZLo2LxBR X-Gm-Gg: AY/fxX5nrRvvd73kzZjJnh0+j/7/WMRufcYgl3verYl1kyY2KjiQnrFlPJh6nhF3ilq 3owESvCIkwU0HXxM5v9dMaghkAaDqdaF1Lv77VdafR5JWqsDW7o6CAxMYMTbC/B15VPe7zPwbHr wZt3VaRU1IIn/xcY9cyUKDmdcgwNrgKl/YEsO28mE4uWHybFUnsSC48A7RfeWmbW+oLvdwmz5ly aPRKYxB6l4RQVWpJXZpS8r/xFxzZ/EBQC2wWYi1VLKOcf6KnlFkxBJk4qattGDEVaVWi6czDPdK jT2NpUBWlq3mWRwMwkH1bipIMCaRY97efGsE2x03YWBRMJeiygVvNDKhlRvp+gauFGm8Ecec8kL RqPaVnsr06cjJhi3nTgkV/qcxNC+UOkqXw6k3k6FbmE+8AHzW6ZttC/sjuhqn6VB/sH5MtYdZRF ny9+0P31KJwCU5TpsR9sU8PsOhu/o6cQd/q2qZ2MXy6nMkYFDe6v5s20HH33ke0rF20g== X-Received: by 2002:a05:600c:8b6c:b0:46e:2815:8568 with SMTP id 5b1f17b1804b1-4801e66fcc5mr130852245e9.10.1768837441407; Mon, 19 Jan 2026 07:44:01 -0800 (PST) Received: from f4d4888f22f2.ant.amazon.com.com ([15.248.2.27]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-47f428ac749sm261155365e9.5.2026.01.19.07.44.00 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Mon, 19 Jan 2026 07:44:01 -0800 (PST) From: Jack Thomson To: mst@redhat.com, david@kernel.org, jasowang@redhat.com Cc: xuanzhuo@linux.alibaba.com, eperezma@redhat.com, virtualization@lists.linux.dev, linux-kernel@vger.kernel.org, kalyazin@amazon.co.uk, xmarcalx@amazon.co.uk, jackabt@amazon.com Subject: [RFC PATCH] virtio_balloon: Support wait on ACK for hinting Date: Mon, 19 Jan 2026 15:42:36 +0000 Message-ID: <20260119154236.39412-1-jackabt.amazon@gmail.com> X-Mailer: git-send-email 2.50.1 Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Jack Thomson This RFC patch adds a new virtio feature for the virtio-balloon driver during free page hinting, which will wait on device ack before committing the range to the free_page_list. The reason for the change is it allows the device to modify this range without it being reclaimed from the free_page_list before the ack is sent. As expected, testing shows this adds overhead to the hinting run duration, increasing it by ~30% with our Firecracker setup. Currently free page hinting is used mainly for live migration, but this would open it up for a new use-case. We would like to leverage this with MADV_DONTNEED to reduce RSS of a guest. We'd like to use hinting because of the flexibility of control it brings compared to reporting, allowing memory to be reclaimed in deterministic periods. The traditional balloon device was tested to be much slower when compared to hinting for these workloads. Currently, without this synchronization, hinted pages may be reclaimed from the free list before the device finishes processing them, making hinting unsuitable for this use-case. Signed-off-by: Jack Thomson --- drivers/virtio/virtio_balloon.c | 21 ++++++++++++++++++--- include/uapi/linux/virtio_balloon.h | 1 + 2 files changed, 19 insertions(+), 3 deletions(-) diff --git a/drivers/virtio/virtio_balloon.c b/drivers/virtio/virtio_balloo= n.c index 74fe59f5a78c..82b560422279 100644 --- a/drivers/virtio/virtio_balloon.c +++ b/drivers/virtio/virtio_balloon.c @@ -596,8 +596,11 @@ static int init_vqs(struct virtio_balloon *vb) vqs_info[VIRTIO_BALLOON_VQ_STATS].callback =3D stats_request; } =20 - if (virtio_has_feature(vb->vdev, VIRTIO_BALLOON_F_FREE_PAGE_HINT)) + if (virtio_has_feature(vb->vdev, VIRTIO_BALLOON_F_FREE_PAGE_HINT)) { vqs_info[VIRTIO_BALLOON_VQ_FREE_PAGE].name =3D "free_page_vq"; + if (virtio_has_feature(vb->vdev, VIRTIO_BALLOON_F_HINT_WAIT_ON_ACK)) + vqs_info[VIRTIO_BALLOON_VQ_FREE_PAGE].callback =3D balloon_ack; + } =20 if (virtio_has_feature(vb->vdev, VIRTIO_BALLOON_F_REPORTING)) { vqs_info[VIRTIO_BALLOON_VQ_REPORTING].name =3D "reporting_vq"; @@ -669,8 +672,11 @@ static int send_cmd_id_start(struct virtio_balloon *vb) virtio_balloon_cmd_id_received(vb)); sg_init_one(&sg, &vb->cmd_id_active, sizeof(vb->cmd_id_active)); err =3D virtqueue_add_outbuf(vq, &sg, 1, &vb->cmd_id_active, GFP_KERNEL); - if (!err) + if (!err) { virtqueue_kick(vq); + if (virtio_has_feature(vb->vdev, VIRTIO_BALLOON_F_HINT_WAIT_ON_ACK)) + wait_event(vb->acked, virtqueue_get_buf(vq, &unused)); + } return err; } =20 @@ -686,8 +692,11 @@ static int send_cmd_id_stop(struct virtio_balloon *vb) =20 sg_init_one(&sg, &vb->cmd_id_stop, sizeof(vb->cmd_id_stop)); err =3D virtqueue_add_outbuf(vq, &sg, 1, &vb->cmd_id_stop, GFP_KERNEL); - if (!err) + if (!err) { virtqueue_kick(vq); + if (virtio_has_feature(vb->vdev, VIRTIO_BALLOON_F_HINT_WAIT_ON_ACK)) + wait_event(vb->acked, virtqueue_get_buf(vq, &unused)); + } return err; } =20 @@ -722,6 +731,8 @@ static int get_free_page_and_send(struct virtio_balloon= *vb) return err; } virtqueue_kick(vq); + if (virtio_has_feature(vb->vdev, VIRTIO_BALLOON_F_HINT_WAIT_ON_ACK)) + wait_event(vb->acked, virtqueue_get_buf(vq, &unused)); spin_lock_irq(&vb->free_page_list_lock); balloon_page_push(&vb->free_page_list, page); vb->num_free_page_blocks++; @@ -1186,6 +1197,9 @@ static int virtballoon_validate(struct virtio_device = *vdev) else if (!virtio_has_feature(vdev, VIRTIO_BALLOON_F_PAGE_POISON)) __virtio_clear_bit(vdev, VIRTIO_BALLOON_F_REPORTING); =20 + if (!virtio_has_feature(vdev, VIRTIO_BALLOON_F_FREE_PAGE_HINT)) + __virtio_clear_bit(vdev, VIRTIO_BALLOON_F_HINT_WAIT_ON_ACK); + __virtio_clear_bit(vdev, VIRTIO_F_ACCESS_PLATFORM); return 0; } @@ -1197,6 +1211,7 @@ static unsigned int features[] =3D { VIRTIO_BALLOON_F_FREE_PAGE_HINT, VIRTIO_BALLOON_F_PAGE_POISON, VIRTIO_BALLOON_F_REPORTING, + VIRTIO_BALLOON_F_HINT_WAIT_ON_ACK, }; =20 static struct virtio_driver virtio_balloon_driver =3D { diff --git a/include/uapi/linux/virtio_balloon.h b/include/uapi/linux/virti= o_balloon.h index ee35a372805d..86698ab06261 100644 --- a/include/uapi/linux/virtio_balloon.h +++ b/include/uapi/linux/virtio_balloon.h @@ -37,6 +37,7 @@ #define VIRTIO_BALLOON_F_FREE_PAGE_HINT 3 /* VQ to report free pages */ #define VIRTIO_BALLOON_F_PAGE_POISON 4 /* Guest is using page poisoning */ #define VIRTIO_BALLOON_F_REPORTING 5 /* Page reporting virtqueue */ +#define VIRTIO_BALLOON_F_HINT_WAIT_ON_ACK 6 /* Page hinting waits on devic= e ack */ =20 /* Size of a PFN in the balloon interface. */ #define VIRTIO_BALLOON_PFN_SHIFT 12 base-commit: 24d479d26b25bce5faea3ddd9fa8f3a6c3129ea7 --=20 2.43.0