From nobody Sat Nov 15 23:41:33 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=yandex-team.ru ARC-Seal: i=1; a=rsa-sha256; t=1744888463; cv=none; d=zohomail.com; s=zohoarc; b=fSrJ+M1gDMNgztQt/pK5zV7C4i9niNqsaGafUNURGjYTj+3+V+LtT7zPiRGY9SBQAHFx0u6bIhe/mhTuz2kknbLvP3PI5dGJ2fZ2nkSxIuyxbjjWh5ydYzltTiCdCiLKqOaeWANmjIANp6eEhXQRRCwnxMoRE3XqDwIGSO+bF0c= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1744888463; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=OFnYkElq7KLQOliisIu7XXuHdcEvLFLPedH92jINYp0=; b=hq+m/6Vo7vhjPNxK4Tk3s+2Cqlu4d/ttfgaFiWqPLqxp8rcpwd5z7+DYoAfaginCYOC7kMO1/8L+tVavk8xDx7hO/8RLEAo6fg8+n+y/haFAvft9HBHciEM59CZFLEU3JyIn+xrZ78SZkMasLOYn32IY1EIGjMG2I7JqbxUQuSI= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1744888463407186.45147081699383; Thu, 17 Apr 2025 04:14:23 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1u5NBe-0003XS-DO; Thu, 17 Apr 2025 07:13:42 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1u5NBc-0003Wj-2S for qemu-devel@nongnu.org; Thu, 17 Apr 2025 07:13:40 -0400 Received: from forwardcorp1a.mail.yandex.net ([178.154.239.72]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1u5NBX-0007AU-HO for qemu-devel@nongnu.org; Thu, 17 Apr 2025 07:13:39 -0400 Received: from mail-nwsmtp-smtp-corp-main-69.vla.yp-c.yandex.net (mail-nwsmtp-smtp-corp-main-69.vla.yp-c.yandex.net [IPv6:2a02:6b8:c15:381b:0:640:f69d:0]) by forwardcorp1a.mail.yandex.net (Yandex) with ESMTPS id 55C1A60AB0; Thu, 17 Apr 2025 14:13:31 +0300 (MSK) Received: from dtalexundeer-nx.yandex-team.ru (unknown [2a02:6b8:b081:8014::1:10]) by mail-nwsmtp-smtp-corp-main-69.vla.yp-c.yandex.net (smtpcorp/Yandex) with ESMTPSA id MDH9PZ2FiKo0-8qlmYwjS; Thu, 17 Apr 2025 14:13:30 +0300 Precedence: bulk X-Yandex-Fwd: 1 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yandex-team.ru; s=default; t=1744888410; bh=OFnYkElq7KLQOliisIu7XXuHdcEvLFLPedH92jINYp0=; h=Message-Id:Date:In-Reply-To:Cc:Subject:References:To:From; b=PV6WmtkucY51a4DtZLPykerG8WZU1ykIdI8VykDW/6pfcy/4ZZqnUst2nXpNjMqY+ MBmfyHxDHsWWRkaQwpL3jLo84SY3xBCK7vDhM4PLyKTNK55CjLRCwkYM/zKY+MkWVR DHue1Qlac2Ji4GXGN1A24sUKo9J+zD0qBw5Pp4FI= Authentication-Results: mail-nwsmtp-smtp-corp-main-69.vla.yp-c.yandex.net; dkim=pass header.i=@yandex-team.ru From: Alexandr Moshkov To: qemu-devel@nongnu.org Cc: Cleber Rosa , "yc-core @ yandex-team . ru" , Paolo Bonzini , =?UTF-8?q?Daniel=20P=20=2E=20Berrang=C3=A9?= , Alexandr Moshkov Subject: [PATCH v5 1/2] tests/functional: add skipLockedMemoryTest decorator Date: Thu, 17 Apr 2025 16:13:20 +0500 Message-Id: <20250417111321.167008-2-dtalexundeer@yandex-team.ru> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20250417111321.167008-1-dtalexundeer@yandex-team.ru> References: <20250417111321.167008-1-dtalexundeer@yandex-team.ru> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=178.154.239.72; envelope-from=dtalexundeer@yandex-team.ru; helo=forwardcorp1a.mail.yandex.net X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, RCVD_IN_VALIDITY_SAFE_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @yandex-team.ru) X-ZM-MESSAGEID: 1744888466428019000 Content-Type: text/plain; charset="utf-8" Used in future commit to skipping execution of a tests if the system's locked memory limit is below the required threshold. Signed-off-by: Alexandr Moshkov --- tests/functional/qemu_test/__init__.py | 2 +- tests/functional/qemu_test/decorators.py | 18 ++++++++++++++++++ 2 files changed, 19 insertions(+), 1 deletion(-) diff --git a/tests/functional/qemu_test/__init__.py b/tests/functional/qemu= _test/__init__.py index af41c2c6a2..6e666a059f 100644 --- a/tests/functional/qemu_test/__init__.py +++ b/tests/functional/qemu_test/__init__.py @@ -15,6 +15,6 @@ from .linuxkernel import LinuxKernelTest from .decorators import skipIfMissingCommands, skipIfNotMachine, \ skipFlakyTest, skipUntrustedTest, skipBigDataTest, skipSlowTest, \ - skipIfMissingImports, skipIfOperatingSystem + skipIfMissingImports, skipIfOperatingSystem, skipLockedMemoryTest from .archive import archive_extract from .uncompress import uncompress diff --git a/tests/functional/qemu_test/decorators.py b/tests/functional/qe= mu_test/decorators.py index 50d29de533..c0d1567b14 100644 --- a/tests/functional/qemu_test/decorators.py +++ b/tests/functional/qemu_test/decorators.py @@ -5,6 +5,7 @@ import importlib import os import platform +import resource from unittest import skipIf, skipUnless =20 from .cmd import which @@ -131,3 +132,20 @@ def skipIfMissingImports(*args): =20 return skipUnless(has_imports, 'required import(s) "%s" not installed'= % ", ".join(args)) + +''' +Decorator to skip execution of a test if the system's +locked memory limit is below the required threshold. +Takes required locked memory threshold in kB. +Example: + + @skipLockedMemoryTest(2_097_152) +''' +def skipLockedMemoryTest(locked_memory): + # get memlock hard limit in bytes + _, ulimit_memory =3D resource.getrlimit(resource.RLIMIT_MEMLOCK) + + return skipUnless( + ulimit_memory =3D=3D resource.RLIM_INFINITY or ulimit_memory >=3D = locked_memory * 1024, + f'Test required {locked_memory} kB of available locked memory', + ) --=20 2.34.1 From nobody Sat Nov 15 23:41:33 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=yandex-team.ru ARC-Seal: i=1; a=rsa-sha256; t=1744888464; cv=none; d=zohomail.com; s=zohoarc; b=G4encDKI0la5Mw1j1rTJ8klELciJIkKiaXYMO4CgBhEZ9e8a3Iq6Wbbw0wM0F6NYZ+QO1Rb68dt+XqAbcHHklzZaMACgB+cCDb5yTjpW3C4Z7o8ZWxx5NXpqS8mltDZvDyT2mDc8jaJqeXFKnXYuBa1dYqdHkNBdEUQ2jrtXREI= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1744888464; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=Mg42Zg1ZyEgQAtkcUl6Dccx4cbgucQhpYSocIFJlCM0=; b=WsUseFMnRcfAwuMZxV0NxT+50MwFsJG0M2wrinlD55iSNrBCYKK/d/izBHdPsKHk5LcB+F1rvUoF9bdjxnE195Dtnt1KR5mx+9toZpdHEkg9a++MPuLSTHQA8gZ1TgEgT2yO6WuNV1p9yh62aVFw1Uquor4JDTxG7XSFn/z3ZOQ= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1744888464320230.08855435780708; Thu, 17 Apr 2025 04:14:24 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1u5NBc-0003X2-Qe; Thu, 17 Apr 2025 07:13:40 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1u5NBa-0003WC-Jf for qemu-devel@nongnu.org; Thu, 17 Apr 2025 07:13:38 -0400 Received: from forwardcorp1a.mail.yandex.net ([2a02:6b8:c0e:500:1:45:d181:df01]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1u5NBX-0007Af-HR for qemu-devel@nongnu.org; Thu, 17 Apr 2025 07:13:38 -0400 Received: from mail-nwsmtp-smtp-corp-main-69.vla.yp-c.yandex.net (mail-nwsmtp-smtp-corp-main-69.vla.yp-c.yandex.net [IPv6:2a02:6b8:c15:381b:0:640:f69d:0]) by forwardcorp1a.mail.yandex.net (Yandex) with ESMTPS id 4AE0260A83; Thu, 17 Apr 2025 14:13:32 +0300 (MSK) Received: from dtalexundeer-nx.yandex-team.ru (unknown [2a02:6b8:b081:8014::1:10]) by mail-nwsmtp-smtp-corp-main-69.vla.yp-c.yandex.net (smtpcorp/Yandex) with ESMTPSA id MDH9PZ2FiKo0-V8zXIySh; Thu, 17 Apr 2025 14:13:31 +0300 Precedence: bulk X-Yandex-Fwd: 1 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yandex-team.ru; s=default; t=1744888411; bh=Mg42Zg1ZyEgQAtkcUl6Dccx4cbgucQhpYSocIFJlCM0=; h=Message-Id:Date:In-Reply-To:Cc:Subject:References:To:From; b=khAAhXu22ZXEbuIz/G99aKBnjGHOcgmoyMuXEDDV4hLPeiEYfdjna9U1Ber5d1cmD e2gdK2kNmAPsIlQg93WVYoM3j9CJBGJZRREKXOPNT/uDNyXQvEGHGtVdqWEYWCH9tE WWuwyNK6Vk4zv8GNK+n33Kbr/UjvbsPawT/ehZN4= Authentication-Results: mail-nwsmtp-smtp-corp-main-69.vla.yp-c.yandex.net; dkim=pass header.i=@yandex-team.ru From: Alexandr Moshkov To: qemu-devel@nongnu.org Cc: Cleber Rosa , "yc-core @ yandex-team . ru" , Paolo Bonzini , =?UTF-8?q?Daniel=20P=20=2E=20Berrang=C3=A9?= , Alexandr Moshkov Subject: [PATCH v5 2/2] tests/functional: add memlock tests Date: Thu, 17 Apr 2025 16:13:21 +0500 Message-Id: <20250417111321.167008-3-dtalexundeer@yandex-team.ru> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20250417111321.167008-1-dtalexundeer@yandex-team.ru> References: <20250417111321.167008-1-dtalexundeer@yandex-team.ru> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=2a02:6b8:c0e:500:1:45:d181:df01; envelope-from=dtalexundeer@yandex-team.ru; helo=forwardcorp1a.mail.yandex.net X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @yandex-team.ru) X-ZM-MESSAGEID: 1744888466488019000 Content-Type: text/plain; charset="utf-8" Add new tests to check the correctness of the `-overcommit memlock` option (possible values: off, on, on-fault) by using `/proc/{qemu_pid}/smaps` file to check in Size, Rss and Locked fields of anonymous segments: * if `memlock=3Doff`, then Locked =3D 0 on every anonymous smaps; * if `memlock=3Don`, then Size, Rss and Locked values must be equal for every anon smaps where Rss is not 0; * if `memlock=3Don-fault`, then Rss and Locked must be equal on every anon smaps and anonymous segment with Rss < Size must exists. Signed-off-by: Alexandr Moshkov --- tests/functional/meson.build | 1 + tests/functional/test_memlock.py | 102 +++++++++++++++++++++++++++++++ 2 files changed, 103 insertions(+) create mode 100755 tests/functional/test_memlock.py diff --git a/tests/functional/meson.build b/tests/functional/meson.build index 0f8be30fe2..339af7835f 100644 --- a/tests/functional/meson.build +++ b/tests/functional/meson.build @@ -61,6 +61,7 @@ tests_generic_system =3D [ 'empty_cpu_model', 'info_usernet', 'version', + 'memlock', ] =20 tests_generic_linuxuser =3D [ diff --git a/tests/functional/test_memlock.py b/tests/functional/test_memlo= ck.py new file mode 100755 index 0000000000..b62f12a715 --- /dev/null +++ b/tests/functional/test_memlock.py @@ -0,0 +1,102 @@ +# Functional test that check overcommit memlock options +# +# Copyright (c) Yandex Technologies LLC, 2025 +# +# Author: +# Alexandr Moshkov +# +# +# This work is licensed under the terms of the GNU GPL, version 2 or +# later. See the COPYING file in the top-level directory. + +import re + +from typing import List, Dict + +from qemu_test import QemuSystemTest +from qemu_test import skipLockedMemoryTest + + +SMAPS_HEADER_PATTERN =3D re.compile(r'^\w+-\w+', re.MULTILINE) +SMAPS_VALUE_PATTERN =3D re.compile(r'^(\w+):\s+(\d+) kB', re.MULTILINE) + + +@skipLockedMemoryTest(2_097_152) # 2GB +class MemlockTest(QemuSystemTest): + """ + Boots a Linux system with memlock options. + Then verify, that this options is working correctly + by checking the smaps of the QEMU proccess. + """ + + def common_vm_setup_with_memlock(self, memlock): + self.vm.add_args('-overcommit', f'mem-lock=3D{memlock}') + self.vm.launch() + + def get_anon_smaps_by_pid(self, pid): + smaps_raw =3D self._get_raw_smaps_by_pid(pid) + return self._parse_anonymous_smaps(smaps_raw) + + def test_memlock_off(self): + self.common_vm_setup_with_memlock('off') + + anon_smaps =3D self.get_anon_smaps_by_pid(self.vm.get_pid()) + + # locked =3D 0 on every smap + for smap in anon_smaps: + self.assertEqual(smap['Locked'], 0) + + def test_memlock_on(self): + self.common_vm_setup_with_memlock('on') + + anon_smaps =3D self.get_anon_smaps_by_pid(self.vm.get_pid()) + + # size =3D rss =3D locked on every smap where rss not 0 + for smap in anon_smaps: + if smap['Rss'] =3D=3D 0: + continue + self.assertTrue(smap['Size'] =3D=3D smap['Rss'] =3D=3D smap['L= ocked']) + + def test_memlock_onfault(self): + self.common_vm_setup_with_memlock('on-fault') + + anon_smaps =3D self.get_anon_smaps_by_pid(self.vm.get_pid()) + + # rss =3D locked on every smap and segment with rss < size exists + exists =3D False + for smap in anon_smaps: + self.assertTrue(smap['Rss'] =3D=3D smap['Locked']) + if smap['Rss'] < smap['Size']: + exists =3D True + self.assertTrue(exists) + + def _parse_anonymous_smaps(self, smaps_raw: str) -> List[Dict[str, int= ]]: + result_segments =3D [] + current_segment =3D {} + is_anonymous =3D False + + for line in smaps_raw.split('\n'): + if SMAPS_HEADER_PATTERN.match(line): + if current_segment and is_anonymous: + result_segments.append(current_segment) + current_segment =3D {} + # anonymous segment header looks like this: + # 7f3b8d3f0000-7f3b8d3f3000 rw-s 00000000 00:0f 1052 + # and non anonymous header looks like this: + # 7f3b8d3f0000-7f3b8d3f3000 rw-s 00000000 00:0f 1052 [st= ack] + is_anonymous =3D len(line.split()) =3D=3D 5 + elif m :=3D SMAPS_VALUE_PATTERN.match(line): + current_segment[m.group(1)] =3D int(m.group(2)) + + if current_segment and is_anonymous: + result_segments.append(current_segment) + + return result_segments + + def _get_raw_smaps_by_pid(self, pid: int) -> str: + with open(f'/proc/{pid}/smaps', 'r') as f: + return f.read() + + +if __name__ =3D=3D '__main__': + MemlockTest.main() --=20 2.34.1