Add new tests to check the correctness of the `-overcommit memlock`
option (possible values: off, on, on-fault) by using
`/proc/{qemu_pid}/smaps` file to check in Size, Rss and Locked fields of
anonymous segments:

* if `memlock=off`, then Locked = 0 on every anonymous smaps;
* if `memlock=on`, then Size, Rss and Locked values must be equal for
every anon smaps where Rss is not 0;
* if `memlock=on-fault`, then Rss and Locked must be equal on every anon
smaps and anonymous segment with Rss < Size must exists.

---

v2 -> v3:
Move tests to tests/functional dir, as the tests/avocado dir is being phased out.
v2 was [PATCH v2] tests/avocado: add memlock tests.
Supersedes: <20250414075702.9248-1-dtalexundeer@yandex-team.ru>

v1 -> v2:
In the previous send, i forgot to specify new patch version (v2)
So i resend previous patch with version specified.

Signed-off-by: Alexandr Moshkov <dtalexundeer@yandex-team.ru>
---
 tests/functional/meson.build     |   1 +
 tests/functional/test_memlock.py | 100 +++++++++++++++++++++++++++++++
 2 files changed, 101 insertions(+)
 create mode 100644 tests/functional/test_memlock.py

diff --git a/tests/functional/meson.build b/tests/functional/meson.build
index 0f8be30fe2..339af7835f 100644
--- a/tests/functional/meson.build
+++ b/tests/functional/meson.build
@@ -61,6 +61,7 @@ tests_generic_system = [
   'empty_cpu_model',
   'info_usernet',
   'version',
+  'memlock',
 ]
 
 tests_generic_linuxuser = [
diff --git a/tests/functional/test_memlock.py b/tests/functional/test_memlock.py
new file mode 100644
index 0000000000..e551e54a77
--- /dev/null
+++ b/tests/functional/test_memlock.py
@@ -0,0 +1,100 @@
+# Functional test that check overcommit memlock options
+#
+# Copyright (c) Yandex Technologies LLC, 2025
+#
+# Author:
+#  Alexandr Moshkov <dtalexundeer@yandex-team.ru>
+#
+#
+# This work is licensed under the terms of the GNU GPL, version 2 or
+# later.  See the COPYING file in the top-level directory.
+
+import re
+
+from typing import List, Dict
+
+from qemu_test import QemuSystemTest
+
+
+SMAPS_HEADER_PATTERN = re.compile(r'^\w+-\w+', re.MULTILINE)
+SMAPS_VALUE_PATTERN = re.compile(r'^(\w+):\s+(\d+) kB', re.MULTILINE)
+
+
+class MemlockTest(QemuSystemTest):
+    """
+    Boots a Linux system with memlock options.
+    Then verify, that this options is working correctly
+    by checking the smaps of the QEMU proccess.
+    """
+
+    def common_vm_setup_with_memlock(self, memlock):
+        self.vm.add_args('-overcommit', f'mem-lock={memlock}')
+        self.vm.launch()
+
+    def get_anon_smaps_by_pid(self, pid):
+        smaps_raw = self._get_raw_smaps_by_pid(pid)
+        return self._parse_anonymous_smaps(smaps_raw)
+
+    def test_memlock_off(self):
+        self.common_vm_setup_with_memlock('off')
+
+        anon_smaps = self.get_anon_smaps_by_pid(self.vm.get_pid())
+
+        # locked = 0 on every smap
+        for smap in anon_smaps:
+            self.assertEqual(smap['Locked'], 0)
+
+    def test_memlock_on(self):
+        self.common_vm_setup_with_memlock('on')
+
+        anon_smaps = self.get_anon_smaps_by_pid(self.vm.get_pid())
+
+        # size = rss = locked on every smap where rss not 0
+        for smap in anon_smaps:
+            if smap['Rss'] == 0:
+                continue
+            self.assertTrue(smap['Size'] == smap['Rss'] == smap['Locked'])
+
+    def test_memlock_onfault(self):
+        self.common_vm_setup_with_memlock('on-fault')
+
+        anon_smaps = self.get_anon_smaps_by_pid(self.vm.get_pid())
+
+        # rss = locked on every smap and segment with rss < size exists
+        exists = False
+        for smap in anon_smaps:
+            self.assertTrue(smap['Rss'] == smap['Locked'])
+            if smap['Rss'] < smap['Size']:
+                exists = True
+        self.assertTrue(exists)
+
+    def _parse_anonymous_smaps(self, smaps_raw: str) -> List[Dict[str, int]]:
+        result_segments = []
+        current_segment = {}
+        is_anonymous = False
+
+        for line in smaps_raw.split('\n'):
+            if SMAPS_HEADER_PATTERN.match(line):
+                if current_segment and is_anonymous:
+                    result_segments.append(current_segment)
+                current_segment = {}
+                # anonymous segment header looks like this:
+                # 7f3b8d3f0000-7f3b8d3f3000 rw-s 00000000 00:0f 1052
+                # and non anonymous header looks like this:
+                # 7f3b8d3f0000-7f3b8d3f3000 rw-s 00000000 00:0f 1052   [stack]
+                is_anonymous = len(line.split()) == 5
+            elif m := SMAPS_VALUE_PATTERN.match(line):
+                current_segment[m.group(1)] = int(m.group(2))
+
+        if current_segment and is_anonymous:
+            result_segments.append(current_segment)
+
+        return result_segments
+
+    def _get_raw_smaps_by_pid(self, pid: int) -> str:
+        with open(f'/proc/{pid}/smaps', 'r') as f:
+            return f.read()
+
+
+if __name__ == '__main__':
+    MemlockTest.main()
-- 
2.34.1

On Tue, Apr 15, 2025 at 02:08:55PM +0500, Alexandr Moshkov wrote:
> Add new tests to check the correctness of the `-overcommit memlock`
> option (possible values: off, on, on-fault) by using
> `/proc/{qemu_pid}/smaps` file to check in Size, Rss and Locked fields of
> anonymous segments:
> 
> * if `memlock=off`, then Locked = 0 on every anonymous smaps;
> * if `memlock=on`, then Size, Rss and Locked values must be equal for
> every anon smaps where Rss is not 0;
> * if `memlock=on-fault`, then Rss and Locked must be equal on every anon
> smaps and anonymous segment with Rss < Size must exists.

How are you running this test ?  Unprivileged users don't get to
lock any non-trivial amount of memory by default, and QEMU functional
tests pretty much exclusively get run as an unprivileged user account.

This test immediately fails when run:

  qemu.machine.machine.VMLaunchFailure: ConnectError: Failed to establish session: EOFError
	Exit code: 1
	Command: ./build/qemu-system-x86_64 -display none -vga none -chardev socket,id=mon,fd=5 -mon chardev=mon,mode=control -overcommit mem-lock=on
	Output: qemu-system-x86_64: mlockall: Cannot allocate memory
        qemu-system-x86_64: locking memory failed

and we don't expect users to run anything as root.

> Signed-off-by: Alexandr Moshkov <dtalexundeer@yandex-team.ru>
> ---
>  tests/functional/meson.build     |   1 +
>  tests/functional/test_memlock.py | 100 +++++++++++++++++++++++++++++++
>  2 files changed, 101 insertions(+)
>  create mode 100644 tests/functional/test_memlock.py

Test files need to have execute permission set.

> 

With regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|

On 4/15/25 14:20, Daniel P. Berrangé wrote:
> On Tue, Apr 15, 2025 at 02:08:55PM +0500, Alexandr Moshkov wrote:
>> Add new tests to check the correctness of the `-overcommit memlock`
>> option (possible values: off, on, on-fault) by using
>> `/proc/{qemu_pid}/smaps` file to check in Size, Rss and Locked fields of
>> anonymous segments:
>>
>> * if `memlock=off`, then Locked = 0 on every anonymous smaps;
>> * if `memlock=on`, then Size, Rss and Locked values must be equal for
>> every anon smaps where Rss is not 0;
>> * if `memlock=on-fault`, then Rss and Locked must be equal on every anon
>> smaps and anonymous segment with Rss < Size must exists.
> How are you running this test ?  Unprivileged users don't get to
> lock any non-trivial amount of memory by default, and QEMU functional
> tests pretty much exclusively get run as an unprivileged user account.
>
> This test immediately fails when run:
>
>    qemu.machine.machine.VMLaunchFailure: ConnectError: Failed to establish session: EOFError
> 	Exit code: 1
> 	Command: ./build/qemu-system-x86_64 -display none -vga none -chardev socket,id=mon,fd=5 -mon chardev=mon,mode=control -overcommit mem-lock=on
> 	Output: qemu-system-x86_64: mlockall: Cannot allocate memory
>          qemu-system-x86_64: locking memory failed
>
> and we don't expect users to run anything as root.
>
Hello, thanks for reply! Looks like i have a larger amount of memory for 
locking in my system:

 > ulimit -l
4063912

I think that's why this test was successfully running on my system.

Honestly,Idon't know yet how to solve this problem properly. Ithink the 
only ways is to runas root (whichis a bad idea)ortoincrease limits on an 
unprivileged user account by using /etc/security/limits.conf.

Best regards,

Alexandr

On Tue, Apr 15, 2025 at 03:27:29PM +0500, Alexandr Moshkov wrote:
> 
> On 4/15/25 14:20, Daniel P. Berrangé wrote:
> > On Tue, Apr 15, 2025 at 02:08:55PM +0500, Alexandr Moshkov wrote:
> > > Add new tests to check the correctness of the `-overcommit memlock`
> > > option (possible values: off, on, on-fault) by using
> > > `/proc/{qemu_pid}/smaps` file to check in Size, Rss and Locked fields of
> > > anonymous segments:
> > > 
> > > * if `memlock=off`, then Locked = 0 on every anonymous smaps;
> > > * if `memlock=on`, then Size, Rss and Locked values must be equal for
> > > every anon smaps where Rss is not 0;
> > > * if `memlock=on-fault`, then Rss and Locked must be equal on every anon
> > > smaps and anonymous segment with Rss < Size must exists.
> > How are you running this test ?  Unprivileged users don't get to
> > lock any non-trivial amount of memory by default, and QEMU functional
> > tests pretty much exclusively get run as an unprivileged user account.
> > 
> > This test immediately fails when run:
> > 
> >    qemu.machine.machine.VMLaunchFailure: ConnectError: Failed to establish session: EOFError
> > 	Exit code: 1
> > 	Command: ./build/qemu-system-x86_64 -display none -vga none -chardev socket,id=mon,fd=5 -mon chardev=mon,mode=control -overcommit mem-lock=on
> > 	Output: qemu-system-x86_64: mlockall: Cannot allocate memory
> >          qemu-system-x86_64: locking memory failed
> > 
> > and we don't expect users to run anything as root.
> > 
> Hello, thanks for reply! Looks like i have a larger amount of memory for
> locking in my system:
> 
> > ulimit -l
> 4063912
> 
> I think that's why this test was successfully running on my system.

In Fedora this limit is set to 8192, and GitLab CI has it set
the same.

> Honestly,Idon't know yet how to solve this problem properly. Ithink the only
> ways is to runas root (whichis a bad idea)ortoincrease limits on an
> unprivileged user account by using /etc/security/limits.conf.

Add a new decorator "skipLockedMemoryTest()" to:

  tests/functional/qemu_test/decorators.py

with the decorator accepting an amount of required locked memory in KB, and
checking it against the current user ulimits. That will make the tst do
the right thing and skip execution of insufficient locked memory is
available.


With regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|