From nobody Fri Apr 4 21:46:49 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=yandex-team.ru ARC-Seal: i=1; a=rsa-sha256; t=1739371250; cv=none; d=zohomail.com; s=zohoarc; b=hu7NhTUhI6t2R3hk9ttkjnTlT315D08nCtGsMga7FaDiq+9pMJxzAvvlazAXF+5od+1kX6US5OND+Po0hyYO5sMciZqtKnns3+5TbyvfT1NJKCSvvPHaXIq0ZbNDqpk7ExWNLYlaw7xiZqVnn3aWAOyVERFSk6bdVNVELLLDBnc= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1739371250; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=ycc6hH2EYxa4YpjcKhOHgF7XnHb2B2jrBSSSENpPfPA=; b=Gmf4ywG+W5Ap4Rja4NIU6vrw6xaPLGgczhJbnM+KoxDG2G9BFtSWYHZlZD3JC++ja7s89DBdha8m5rLJxc+9U7NTxfE5xqIde+mbz6WG6wCuCoJnAsH4tKm1GNT3kTDvbScTz6g/4j7skvpCIrMlSuUGTW1q0FA6moQozPoHeNg= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1739371250031311.7090298148888; Wed, 12 Feb 2025 06:40:50 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1tiDu6-00027I-SX; Wed, 12 Feb 2025 09:39:54 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tiDu4-00026d-A2 for qemu-devel@nongnu.org; Wed, 12 Feb 2025 09:39:52 -0500 Received: from forwardcorp1b.mail.yandex.net ([178.154.239.136]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tiDu2-00080U-Di for qemu-devel@nongnu.org; Wed, 12 Feb 2025 09:39:51 -0500 Received: from mail-nwsmtp-smtp-corp-canary-81.sas.yp-c.yandex.net (mail-nwsmtp-smtp-corp-canary-81.sas.yp-c.yandex.net [IPv6:2a02:6b8:c10:4a4:0:640:7b31:0]) by forwardcorp1b.mail.yandex.net (Yandex) with ESMTPS id 5016060D7D; Wed, 12 Feb 2025 17:39:47 +0300 (MSK) Received: from d-tatianin-lin.yandex-team.ru (unknown [2a02:6b8:b081:b518::1:2f]) by mail-nwsmtp-smtp-corp-canary-81.sas.yp-c.yandex.net (smtpcorp/Yandex) with ESMTPSA id ddQHZB0Id8c0-4FitjubQ; Wed, 12 Feb 2025 17:39:46 +0300 X-Yandex-Fwd: 1 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yandex-team.ru; s=default; t=1739371186; bh=ycc6hH2EYxa4YpjcKhOHgF7XnHb2B2jrBSSSENpPfPA=; h=Message-Id:Date:In-Reply-To:Cc:Subject:References:To:From; b=lZ01fUfGovtA6cL5nBGWWD19GUurpXpfqiPlHq0VXr//6lijYDYn+zxipfbKK5uzd 4UhN6Xtq9llhacXf4RaxSTnclloz/56w6psw0GFSL2oPZnuVjIBmicbY4CmH8/5jX9 MfieIXIemIJ8pCq70uEzwcfaK+RJV90U8X/0pXCM= Authentication-Results: mail-nwsmtp-smtp-corp-canary-81.sas.yp-c.yandex.net; dkim=pass header.i=@yandex-team.ru From: Daniil Tatianin To: Paolo Bonzini Cc: Daniil Tatianin , Stefan Weil , Peter Xu , Fabiano Rosas , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , Peter Maydell , qemu-devel@nongnu.org Subject: [PATCH v5 1/4] os: add an ability to lock memory on_fault Date: Wed, 12 Feb 2025 17:39:17 +0300 Message-Id: <20250212143920.1269754-2-d-tatianin@yandex-team.ru> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20250212143920.1269754-1-d-tatianin@yandex-team.ru> References: <20250212143920.1269754-1-d-tatianin@yandex-team.ru> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=178.154.239.136; envelope-from=d-tatianin@yandex-team.ru; helo=forwardcorp1b.mail.yandex.net X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_VALIDITY_CERTIFIED_BLOCKED=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @yandex-team.ru) X-ZM-MESSAGEID: 1739371251086019000 Content-Type: text/plain; charset="utf-8" This will be used in the following commits to make it possible to only lock memory on fault instead of right away. Signed-off-by: Daniil Tatianin Reviewed-by: Vladimir Sementsov-Ogievskiy --- include/system/os-posix.h | 2 +- include/system/os-win32.h | 3 ++- meson.build | 6 ++++++ migration/postcopy-ram.c | 2 +- os-posix.c | 14 ++++++++++++-- system/vl.c | 2 +- 6 files changed, 23 insertions(+), 6 deletions(-) diff --git a/include/system/os-posix.h b/include/system/os-posix.h index b881ac6c6f..ce5b3bccf8 100644 --- a/include/system/os-posix.h +++ b/include/system/os-posix.h @@ -53,7 +53,7 @@ bool os_set_runas(const char *user_id); void os_set_chroot(const char *path); void os_setup_limits(void); void os_setup_post(void); -int os_mlock(void); +int os_mlock(bool on_fault); =20 /** * qemu_alloc_stack: diff --git a/include/system/os-win32.h b/include/system/os-win32.h index b82a5d3ad9..cd61d69e10 100644 --- a/include/system/os-win32.h +++ b/include/system/os-win32.h @@ -123,8 +123,9 @@ static inline bool is_daemonized(void) return false; } =20 -static inline int os_mlock(void) +static inline int os_mlock(bool on_fault) { + (void)on_fault; return -ENOSYS; } =20 diff --git a/meson.build b/meson.build index 18cf9e2913..59953cbe6b 100644 --- a/meson.build +++ b/meson.build @@ -2885,6 +2885,12 @@ config_host_data.set('HAVE_MLOCKALL', cc.links(gnu_s= ource_prefix + ''' return mlockall(MCL_FUTURE); }''')) =20 +config_host_data.set('HAVE_MLOCK_ONFAULT', cc.links(gnu_source_prefix + ''' + #include + int main(void) { + return mlockall(MCL_FUTURE | MCL_ONFAULT); + }''')) + have_l2tpv3 =3D false if get_option('l2tpv3').allowed() and have_system have_l2tpv3 =3D cc.has_type('struct mmsghdr', diff --git a/migration/postcopy-ram.c b/migration/postcopy-ram.c index 6a6da6ba7f..fc4d8a10df 100644 --- a/migration/postcopy-ram.c +++ b/migration/postcopy-ram.c @@ -652,7 +652,7 @@ int postcopy_ram_incoming_cleanup(MigrationIncomingStat= e *mis) } =20 if (enable_mlock) { - if (os_mlock() < 0) { + if (os_mlock(false) < 0) { error_report("mlock: %s", strerror(errno)); /* * It doesn't feel right to fail at this point, we have a valid diff --git a/os-posix.c b/os-posix.c index 9cce55ff2f..17b144c0a2 100644 --- a/os-posix.c +++ b/os-posix.c @@ -327,18 +327,28 @@ void os_set_line_buffering(void) setvbuf(stdout, NULL, _IOLBF, 0); } =20 -int os_mlock(void) +int os_mlock(bool on_fault) { #ifdef HAVE_MLOCKALL int ret =3D 0; + int flags =3D MCL_CURRENT | MCL_FUTURE; =20 - ret =3D mlockall(MCL_CURRENT | MCL_FUTURE); +#ifdef HAVE_MLOCK_ONFAULT + if (on_fault) { + flags |=3D MCL_ONFAULT; + } +#else + (void)on_fault; +#endif + + ret =3D mlockall(flags); if (ret < 0) { error_report("mlockall: %s", strerror(errno)); } =20 return ret; #else + (void)on_fault; return -ENOSYS; #endif } diff --git a/system/vl.c b/system/vl.c index 9c6942c6cf..e94fc7ea35 100644 --- a/system/vl.c +++ b/system/vl.c @@ -797,7 +797,7 @@ static QemuOptsList qemu_run_with_opts =3D { static void realtime_init(void) { if (enable_mlock) { - if (os_mlock() < 0) { + if (os_mlock(false) < 0) { error_report("locking memory failed"); exit(1); } --=20 2.34.1 From nobody Fri Apr 4 21:46:49 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=yandex-team.ru ARC-Seal: i=1; a=rsa-sha256; t=1739371210; cv=none; d=zohomail.com; s=zohoarc; b=hMgSWxjjUuaUmf0iT29iO5du1KaaWGcITHCVhnXX/+wOTGEl80pZLvl7F/w8kbXl8x+4GM5qXMReLDmsJf59APuPdfMdQBarpwkMcXQdPhGdEkCTAQTTqOk6ng2pBiTkNhm87GVqT6ieVabdkR6BuiQ56v302+alHfNo28d6hLs= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1739371210; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=ybIkw9tiflFmhaHlKoA26s39xuP3YJHyyhxeEkdpcZU=; b=f1x3RHBkilqJSFYs1Ay5ZtHKLL1bqZ0JsZZqA46MW7ZpixTLnRi3teXIx3VAOp824HlX15z8gNIyTDJj6qne7z70BWlGEH883T9wJgye6p22m3aFYU4qg6YjZc6a8mTvKIfehRWELjD/XkmAKOyiN3iuq9753SX88Bge56oeOhg= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1739371210522910.5480092392526; Wed, 12 Feb 2025 06:40:10 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1tiDu8-00027n-PF; Wed, 12 Feb 2025 09:39:57 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tiDu5-000275-In for qemu-devel@nongnu.org; Wed, 12 Feb 2025 09:39:53 -0500 Received: from forwardcorp1b.mail.yandex.net ([2a02:6b8:c02:900:1:45:d181:df01]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tiDu3-00080n-3h for qemu-devel@nongnu.org; Wed, 12 Feb 2025 09:39:53 -0500 Received: from mail-nwsmtp-smtp-corp-canary-81.sas.yp-c.yandex.net (mail-nwsmtp-smtp-corp-canary-81.sas.yp-c.yandex.net [IPv6:2a02:6b8:c10:4a4:0:640:7b31:0]) by forwardcorp1b.mail.yandex.net (Yandex) with ESMTPS id B509A611E2; Wed, 12 Feb 2025 17:39:49 +0300 (MSK) Received: from d-tatianin-lin.yandex-team.ru (unknown [2a02:6b8:b081:b518::1:2f]) by mail-nwsmtp-smtp-corp-canary-81.sas.yp-c.yandex.net (smtpcorp/Yandex) with ESMTPSA id ddQHZB0Id8c0-uma463PO; Wed, 12 Feb 2025 17:39:49 +0300 X-Yandex-Fwd: 1 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yandex-team.ru; s=default; t=1739371189; bh=ybIkw9tiflFmhaHlKoA26s39xuP3YJHyyhxeEkdpcZU=; h=Message-Id:Date:In-Reply-To:Cc:Subject:References:To:From; b=ut3oeivrSCZq1DF+AZlFqIIremfC5Ex0T+chWpsIsf4WCt1S7Z0L1vTBDKOcY4yO7 Ke8xcQ8Y/I7y3KJSLMq+BYUxtSOkk+4ZlUbune2o7FrD/RILNuhqbbsDVQK1ASCOzG 2tkdgwdUC5qDYMoO6NpW0UtQGVKba6/zlRPJoA8U= Authentication-Results: mail-nwsmtp-smtp-corp-canary-81.sas.yp-c.yandex.net; dkim=pass header.i=@yandex-team.ru From: Daniil Tatianin To: Paolo Bonzini Cc: Daniil Tatianin , Stefan Weil , Peter Xu , Fabiano Rosas , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , Peter Maydell , qemu-devel@nongnu.org, Vladimir Sementsov-Ogievskiy Subject: [PATCH v5 2/4] system/vl: extract overcommit option parsing into a helper Date: Wed, 12 Feb 2025 17:39:18 +0300 Message-Id: <20250212143920.1269754-3-d-tatianin@yandex-team.ru> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20250212143920.1269754-1-d-tatianin@yandex-team.ru> References: <20250212143920.1269754-1-d-tatianin@yandex-team.ru> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=2a02:6b8:c02:900:1:45:d181:df01; envelope-from=d-tatianin@yandex-team.ru; helo=forwardcorp1b.mail.yandex.net X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @yandex-team.ru) X-ZM-MESSAGEID: 1739371212686019100 Content-Type: text/plain; charset="utf-8" This will be extended in the future commits, let's move it out of line right away so that it's easier to read. Reviewed-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Peter Xu Signed-off-by: Daniil Tatianin --- system/vl.c | 21 ++++++++++++++------- 1 file changed, 14 insertions(+), 7 deletions(-) diff --git a/system/vl.c b/system/vl.c index e94fc7ea35..72a40985f5 100644 --- a/system/vl.c +++ b/system/vl.c @@ -1875,6 +1875,19 @@ static void object_option_parse(const char *str) visit_free(v); } =20 +static void overcommit_parse(const char *str) +{ + QemuOpts *opts; + + opts =3D qemu_opts_parse_noisily(qemu_find_opts("overcommit"), + str, false); + if (!opts) { + exit(1); + } + enable_mlock =3D qemu_opt_get_bool(opts, "mem-lock", enable_mlock); + enable_cpu_pm =3D qemu_opt_get_bool(opts, "cpu-pm", enable_cpu_pm); +} + /* * Very early object creation, before the sandbox options have been activa= ted. */ @@ -3575,13 +3588,7 @@ void qemu_init(int argc, char **argv) object_option_parse(optarg); break; case QEMU_OPTION_overcommit: - opts =3D qemu_opts_parse_noisily(qemu_find_opts("overcommi= t"), - optarg, false); - if (!opts) { - exit(1); - } - enable_mlock =3D qemu_opt_get_bool(opts, "mem-lock", enabl= e_mlock); - enable_cpu_pm =3D qemu_opt_get_bool(opts, "cpu-pm", enable= _cpu_pm); + overcommit_parse(optarg); break; case QEMU_OPTION_compat: { --=20 2.34.1 From nobody Fri Apr 4 21:46:49 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=yandex-team.ru ARC-Seal: i=1; a=rsa-sha256; t=1739371262; cv=none; d=zohomail.com; s=zohoarc; b=oICQbEXt/V2GiY3IAYJyu5EP7UsR8+hWVJyx8aOpaH/NvfXpxXUVUeEy2wPK8HyF/+bKSnjPQ8bAfN4yEq3yyNm9QOf5PwOLY1j2BQfjWj00cdLQWAq/wjq8M/f5JRslPF6b1SUKi8CdzFb+E7Ggkb6xLY9bhMtfa34x8eQaQbI= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1739371262; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=LHVJkeRgJzCWlwQJpNDZyKo0KBJKb/s9XQXyF5ZbvLA=; b=iZkpt2PSTnpU3TJ00IsbwbTmSaIjFJNKwZFGaOFdidiVY62DlEtOCGexSHc2OtrS4K+1BZFe+eLDdZK5j7aR9yp2ZrbbL1nLvPGkxzyi7hb+xKgUMVO7RRv2bQofvmayd2i/vhgBj91wX8wAXJJa8COXgv+ElCnjJR2VaXC97tM= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1739371261972216.29553599502356; Wed, 12 Feb 2025 06:41:01 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1tiDuC-000293-Hw; Wed, 12 Feb 2025 09:40:00 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tiDu7-00027f-BM for qemu-devel@nongnu.org; Wed, 12 Feb 2025 09:39:55 -0500 Received: from forwardcorp1b.mail.yandex.net ([178.154.239.136]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tiDu4-000811-Ut for qemu-devel@nongnu.org; Wed, 12 Feb 2025 09:39:54 -0500 Received: from mail-nwsmtp-smtp-corp-canary-81.sas.yp-c.yandex.net (mail-nwsmtp-smtp-corp-canary-81.sas.yp-c.yandex.net [IPv6:2a02:6b8:c10:4a4:0:640:7b31:0]) by forwardcorp1b.mail.yandex.net (Yandex) with ESMTPS id 69ABB6117D; Wed, 12 Feb 2025 17:39:51 +0300 (MSK) Received: from d-tatianin-lin.yandex-team.ru (unknown [2a02:6b8:b081:b518::1:2f]) by mail-nwsmtp-smtp-corp-canary-81.sas.yp-c.yandex.net (smtpcorp/Yandex) with ESMTPSA id ddQHZB0Id8c0-6ZYBEN7u; Wed, 12 Feb 2025 17:39:50 +0300 X-Yandex-Fwd: 1 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yandex-team.ru; s=default; t=1739371190; bh=LHVJkeRgJzCWlwQJpNDZyKo0KBJKb/s9XQXyF5ZbvLA=; h=Message-Id:Date:In-Reply-To:Cc:Subject:References:To:From; b=pF7YDQLURKsMuOAryQKWU8UAEX0tY74/qO0u15kipoLw0e/ypMyEvbVRonuYd+uOb xjhV18S8pmcNJhY6U8vq8FJPCQf0k0zH62dO7T/NH4qn+vSfodEKDGuJYph1WS1xYe BRH3Ml1wrS+trWV8YXeLeZJ4msWZZTViBboCyyBg= Authentication-Results: mail-nwsmtp-smtp-corp-canary-81.sas.yp-c.yandex.net; dkim=pass header.i=@yandex-team.ru From: Daniil Tatianin To: Paolo Bonzini Cc: Daniil Tatianin , Stefan Weil , Peter Xu , Fabiano Rosas , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , Peter Maydell , qemu-devel@nongnu.org, Vladimir Sementsov-Ogievskiy Subject: [PATCH v5 3/4] system: introduce a new MlockState enum Date: Wed, 12 Feb 2025 17:39:19 +0300 Message-Id: <20250212143920.1269754-4-d-tatianin@yandex-team.ru> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20250212143920.1269754-1-d-tatianin@yandex-team.ru> References: <20250212143920.1269754-1-d-tatianin@yandex-team.ru> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=178.154.239.136; envelope-from=d-tatianin@yandex-team.ru; helo=forwardcorp1b.mail.yandex.net X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_VALIDITY_CERTIFIED_BLOCKED=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @yandex-team.ru) X-ZM-MESSAGEID: 1739371264378019100 Content-Type: text/plain; charset="utf-8" Replace the boolean value enable_mlock with an enum and add a helper to decide whether we should be calling os_mlock. This is a stepping stone towards introducing a new mlock mode, which will be the third possible state of this enum. Reviewed-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Peter Xu Signed-off-by: Daniil Tatianin --- hw/virtio/virtio-mem.c | 2 +- include/system/system.h | 10 +++++++++- migration/postcopy-ram.c | 2 +- system/globals.c | 7 ++++++- system/vl.c | 9 +++++++-- 5 files changed, 24 insertions(+), 6 deletions(-) diff --git a/hw/virtio/virtio-mem.c b/hw/virtio/virtio-mem.c index b1a003736b..7b140add76 100644 --- a/hw/virtio/virtio-mem.c +++ b/hw/virtio/virtio-mem.c @@ -991,7 +991,7 @@ static void virtio_mem_device_realize(DeviceState *dev,= Error **errp) return; } =20 - if (enable_mlock) { + if (should_mlock(mlock_state)) { error_setg(errp, "Incompatible with mlock"); return; } diff --git a/include/system/system.h b/include/system/system.h index 0cbb43ec30..dc7628357a 100644 --- a/include/system/system.h +++ b/include/system/system.h @@ -44,10 +44,18 @@ extern int display_opengl; extern const char *keyboard_layout; extern int old_param; extern uint8_t *boot_splash_filedata; -extern bool enable_mlock; extern bool enable_cpu_pm; extern QEMUClockType rtc_clock; =20 +typedef enum { + MLOCK_OFF =3D 0, + MLOCK_ON, +} MlockState; + +bool should_mlock(MlockState); + +extern MlockState mlock_state; + #define MAX_OPTION_ROMS 16 typedef struct QEMUOptionRom { const char *name; diff --git a/migration/postcopy-ram.c b/migration/postcopy-ram.c index fc4d8a10df..04068ee039 100644 --- a/migration/postcopy-ram.c +++ b/migration/postcopy-ram.c @@ -651,7 +651,7 @@ int postcopy_ram_incoming_cleanup(MigrationIncomingStat= e *mis) mis->have_fault_thread =3D false; } =20 - if (enable_mlock) { + if (should_mlock(mlock_state)) { if (os_mlock(false) < 0) { error_report("mlock: %s", strerror(errno)); /* diff --git a/system/globals.c b/system/globals.c index 4867c93ca6..adeff38348 100644 --- a/system/globals.c +++ b/system/globals.c @@ -31,10 +31,15 @@ #include "system/cpus.h" #include "system/system.h" =20 +bool should_mlock(MlockState state) +{ + return state =3D=3D MLOCK_ON; +} + enum vga_retrace_method vga_retrace_method =3D VGA_RETRACE_DUMB; int display_opengl; const char* keyboard_layout; -bool enable_mlock; +MlockState mlock_state; bool enable_cpu_pm; int autostart =3D 1; int vga_interface_type =3D VGA_NONE; diff --git a/system/vl.c b/system/vl.c index 72a40985f5..2895824c1a 100644 --- a/system/vl.c +++ b/system/vl.c @@ -796,7 +796,7 @@ static QemuOptsList qemu_run_with_opts =3D { =20 static void realtime_init(void) { - if (enable_mlock) { + if (should_mlock(mlock_state)) { if (os_mlock(false) < 0) { error_report("locking memory failed"); exit(1); @@ -1878,13 +1878,18 @@ static void object_option_parse(const char *str) static void overcommit_parse(const char *str) { QemuOpts *opts; + bool enable_mlock; =20 opts =3D qemu_opts_parse_noisily(qemu_find_opts("overcommit"), str, false); if (!opts) { exit(1); } - enable_mlock =3D qemu_opt_get_bool(opts, "mem-lock", enable_mlock); + + enable_mlock =3D qemu_opt_get_bool(opts, "mem-lock", + should_mlock(mlock_state)); + mlock_state =3D enable_mlock ? MLOCK_ON : MLOCK_OFF; + enable_cpu_pm =3D qemu_opt_get_bool(opts, "cpu-pm", enable_cpu_pm); } =20 --=20 2.34.1 From nobody Fri Apr 4 21:46:49 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=yandex-team.ru ARC-Seal: i=1; a=rsa-sha256; t=1739371248; cv=none; d=zohomail.com; s=zohoarc; b=Ukues+hA47ADGyeUeL1nfhoCkqnoC9ml19gN/GxpZNgMDiLVuXhDWLjUjuahb0JXWUgls4tVioG3QPU2ucNDNFU6rrW8UoFL7zOT4+zVPqFi9542rTUqB2GtlP1Ta5nJM1SA6DXbBseTvr8NtHeyTPmCmg7FFz+FqgNv7hviY7Y= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1739371248; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=RpcjKs5rWaYLMB4QC5HMd5w0Oo5G6GbkQbmSzu3kH0I=; b=MRLeMTI0cSz9n5/TYKJLxWO66nhc2p9pCsnRqmSrNXqIzjBCPFXE7vOKfnLuw1K91EkS4OsZWSMTRQCAXQz3F0noLbhXfYqL3CJiJ5FmRm4fLUdNNoswLAQH3gyax91IXgdWF8MEM0KxhPDqEZUNq/MHeX87qeb7r7Ev7fd4FQc= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1739371248482505.35483543142277; Wed, 12 Feb 2025 06:40:48 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1tiDuD-00029I-4x; Wed, 12 Feb 2025 09:40:01 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tiDu8-00027w-RL for qemu-devel@nongnu.org; Wed, 12 Feb 2025 09:39:57 -0500 Received: from forwardcorp1b.mail.yandex.net ([178.154.239.136]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tiDu6-00081V-Jy for qemu-devel@nongnu.org; Wed, 12 Feb 2025 09:39:56 -0500 Received: from mail-nwsmtp-smtp-corp-canary-81.sas.yp-c.yandex.net (mail-nwsmtp-smtp-corp-canary-81.sas.yp-c.yandex.net [IPv6:2a02:6b8:c10:4a4:0:640:7b31:0]) by forwardcorp1b.mail.yandex.net (Yandex) with ESMTPS id 3FEFF61179; Wed, 12 Feb 2025 17:39:53 +0300 (MSK) Received: from d-tatianin-lin.yandex-team.ru (unknown [2a02:6b8:b081:b518::1:2f]) by mail-nwsmtp-smtp-corp-canary-81.sas.yp-c.yandex.net (smtpcorp/Yandex) with ESMTPSA id ddQHZB0Id8c0-8c4ucby6; Wed, 12 Feb 2025 17:39:52 +0300 X-Yandex-Fwd: 1 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yandex-team.ru; s=default; t=1739371192; bh=RpcjKs5rWaYLMB4QC5HMd5w0Oo5G6GbkQbmSzu3kH0I=; h=Message-Id:Date:In-Reply-To:Cc:Subject:References:To:From; b=dHxaELCIhn3YTialyzUnVcdWS4Q1fc8MrTOQ4XF5bi8qOMtHjmbkDKqh/YltJTfM/ GNojVoAIlJ5s1Rke8NX69+RMg5G9u67WKjBO3Jp10Gwfrw3G5JHoic8Zet1y5O85hm EQaFAEvjfE6Gc4NbdepNqHvpmEON2olj6aoje32c= Authentication-Results: mail-nwsmtp-smtp-corp-canary-81.sas.yp-c.yandex.net; dkim=pass header.i=@yandex-team.ru From: Daniil Tatianin To: Paolo Bonzini Cc: Daniil Tatianin , Stefan Weil , Peter Xu , Fabiano Rosas , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , Peter Maydell , qemu-devel@nongnu.org, Vladimir Sementsov-Ogievskiy Subject: [PATCH v5 4/4] overcommit: introduce mem-lock=on-fault Date: Wed, 12 Feb 2025 17:39:20 +0300 Message-Id: <20250212143920.1269754-5-d-tatianin@yandex-team.ru> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20250212143920.1269754-1-d-tatianin@yandex-team.ru> References: <20250212143920.1269754-1-d-tatianin@yandex-team.ru> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=178.154.239.136; envelope-from=d-tatianin@yandex-team.ru; helo=forwardcorp1b.mail.yandex.net X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_VALIDITY_CERTIFIED_BLOCKED=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @yandex-team.ru) X-ZM-MESSAGEID: 1739371249138019000 Content-Type: text/plain; charset="utf-8" Locking the memory without MCL_ONFAULT instantly prefaults any mmaped anonymous memory with a write-fault, which introduces a lot of extra overhead in terms of memory usage when all you want to do is to prevent kcompactd from migrating and compacting QEMU pages. Add an option to only lock pages lazily as they're faulted by the process by using MCL_ONFAULT if asked. Reviewed-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Peter Xu Signed-off-by: Daniil Tatianin --- include/system/system.h | 2 ++ migration/postcopy-ram.c | 2 +- qemu-options.hx | 14 +++++++++----- system/globals.c | 7 ++++++- system/vl.c | 34 +++++++++++++++++++++++++++------- 5 files changed, 45 insertions(+), 14 deletions(-) diff --git a/include/system/system.h b/include/system/system.h index dc7628357a..a7effe7dfd 100644 --- a/include/system/system.h +++ b/include/system/system.h @@ -50,9 +50,11 @@ extern QEMUClockType rtc_clock; typedef enum { MLOCK_OFF =3D 0, MLOCK_ON, + MLOCK_ON_FAULT, } MlockState; =20 bool should_mlock(MlockState); +bool is_mlock_on_fault(MlockState); =20 extern MlockState mlock_state; =20 diff --git a/migration/postcopy-ram.c b/migration/postcopy-ram.c index 04068ee039..5d3edfcfec 100644 --- a/migration/postcopy-ram.c +++ b/migration/postcopy-ram.c @@ -652,7 +652,7 @@ int postcopy_ram_incoming_cleanup(MigrationIncomingStat= e *mis) } =20 if (should_mlock(mlock_state)) { - if (os_mlock(false) < 0) { + if (os_mlock(is_mlock_on_fault(mlock_state)) < 0) { error_report("mlock: %s", strerror(errno)); /* * It doesn't feel right to fail at this point, we have a valid diff --git a/qemu-options.hx b/qemu-options.hx index 1b26ad53bd..61270e3206 100644 --- a/qemu-options.hx +++ b/qemu-options.hx @@ -4632,21 +4632,25 @@ SRST ERST =20 DEF("overcommit", HAS_ARG, QEMU_OPTION_overcommit, - "-overcommit [mem-lock=3Don|off][cpu-pm=3Don|off]\n" + "-overcommit [mem-lock=3Don|off|on-fault][cpu-pm=3Don|off]\n" " run qemu with overcommit hints\n" - " mem-lock=3Don|off controls memory lock support (defau= lt: off)\n" + " mem-lock=3Don|off|on-fault controls memory lock suppo= rt (default: off)\n" " cpu-pm=3Don|off controls cpu power management (defaul= t: off)\n", QEMU_ARCH_ALL) SRST -``-overcommit mem-lock=3Don|off`` +``-overcommit mem-lock=3Don|off|on-fault`` \=20 ``-overcommit cpu-pm=3Don|off`` Run qemu with hints about host resource overcommit. The default is to assume that host overcommits all resources. =20 Locking qemu and guest memory can be enabled via ``mem-lock=3Don`` - (disabled by default). This works when host memory is not - overcommitted and reduces the worst-case latency for guest. + or ``mem-lock=3Don-fault`` (disabled by default). This works when + host memory is not overcommitted and reduces the worst-case latency for + guest. The on-fault option is better for reducing the memory footprint + since it makes allocations lazy, but the pages still get locked in pla= ce + once faulted by the guest or QEMU. Note that the two options are mutua= lly + exclusive. =20 Guest ability to manage power state of host cpus (increasing latency for other processes on the same host cpu, but decreasing latency for diff --git a/system/globals.c b/system/globals.c index adeff38348..316623bd20 100644 --- a/system/globals.c +++ b/system/globals.c @@ -33,7 +33,12 @@ =20 bool should_mlock(MlockState state) { - return state =3D=3D MLOCK_ON; + return state =3D=3D MLOCK_ON || state =3D=3D MLOCK_ON_FAULT; +} + +bool is_mlock_on_fault(MlockState state) +{ + return state =3D=3D MLOCK_ON_FAULT; } =20 enum vga_retrace_method vga_retrace_method =3D VGA_RETRACE_DUMB; diff --git a/system/vl.c b/system/vl.c index 2895824c1a..3c0fa2ff64 100644 --- a/system/vl.c +++ b/system/vl.c @@ -351,7 +351,7 @@ static QemuOptsList qemu_overcommit_opts =3D { .desc =3D { { .name =3D "mem-lock", - .type =3D QEMU_OPT_BOOL, + .type =3D QEMU_OPT_STRING, }, { .name =3D "cpu-pm", @@ -797,7 +797,7 @@ static QemuOptsList qemu_run_with_opts =3D { static void realtime_init(void) { if (should_mlock(mlock_state)) { - if (os_mlock(false) < 0) { + if (os_mlock(is_mlock_on_fault(mlock_state)) < 0) { error_report("locking memory failed"); exit(1); } @@ -1878,7 +1878,7 @@ static void object_option_parse(const char *str) static void overcommit_parse(const char *str) { QemuOpts *opts; - bool enable_mlock; + const char *mem_lock_opt; =20 opts =3D qemu_opts_parse_noisily(qemu_find_opts("overcommit"), str, false); @@ -1886,11 +1886,31 @@ static void overcommit_parse(const char *str) exit(1); } =20 - enable_mlock =3D qemu_opt_get_bool(opts, "mem-lock", - should_mlock(mlock_state)); - mlock_state =3D enable_mlock ? MLOCK_ON : MLOCK_OFF; - enable_cpu_pm =3D qemu_opt_get_bool(opts, "cpu-pm", enable_cpu_pm); + + mem_lock_opt =3D qemu_opt_get(opts, "mem-lock"); + if (!mem_lock_opt) { + return; + } + + if (strcmp(mem_lock_opt, "on") =3D=3D 0) { + mlock_state =3D MLOCK_ON; + return; + } + + if (strcmp(mem_lock_opt, "off") =3D=3D 0) { + mlock_state =3D MLOCK_OFF; + return; + } + + if (strcmp(mem_lock_opt, "on-fault") =3D=3D 0) { + mlock_state =3D MLOCK_ON_FAULT; + return; + } + + error_report("parameter 'mem-lock' expects one of " + "'on', 'off', 'on-fault'"); + exit(1); } =20 /* --=20 2.34.1