[PATCH 0/7] tests/qtest: fix the disk tests on a host with a small /tmp

Denis V. Lunev posted 7 patches 1 month ago
Only 0 patches received!
tests/qtest/ahci-test.c        |  48 ++++++++---
tests/qtest/hd-geo-test.c      | 145 ++++++++++++++++++++++-----------
tests/qtest/ide-test.c         |  96 +++++++++++++++++-----
tests/qtest/libqos/libqos.c    |   4 +-
tests/qtest/libqos/libqos.h    |   2 +-
tests/qtest/libqtest.c         |   9 +-
tests/qtest/libqtest.h         |  12 +++
tests/qtest/virtio-scsi-test.c |   5 +-
8 files changed, 237 insertions(+), 84 deletions(-)
[PATCH 0/7] tests/qtest: fix the disk tests on a host with a small /tmp
Posted by Denis V. Lunev 1 month ago
John reported ide-test dying at startup in the NetBSD VM:

  ERROR:../src/tests/qtest/ide-test.c:1260:main: assertion failed:
  (ret == 0)

The hypothesys is that the ftruncate() of a 64 MiB scratch image. NetBSD
mounts /tmp as a tmpfs sized at 25% of RAM and charges a file its full
length themoment it is extended, so a sparse image is not free there and
the call returns ENOSPC. The assert dates to 2013, nothing regressed.

The series is designed to do two things:
* reduce the amount of space needed switching to QCOW2
* tolerate the failure skipping the test if there is not enough space

Patches 1-4 skip instead of aborting, the way hd-geo-test has since
a0d6d7454a. Patches 5-7 build the images with qemu-img where only the
virtual size matters: peak scratch space goes from 129 MiB to 1.6 MiB
for ide-test and from 4126 MiB to 1.5 MiB for hd-geo-test.

Hope this should help. John, can you give a try to this in the test?
I do not have NetBSD handy and will start with this tomorrow if you
will share guest config.

Cc: John Snow <jsnow@redhat.com>
Cc: Thomas Huth <thuth@redhat.com>
Signed-off-by: Denis V. Lunev <den@openvz.org>

Denis V. Lunev (7):
  tests/qtest/libqos: let mkqcow2() report failure
  tests/qtest/ide-test: skip when the scratch files cannot be created
  tests/qtest/ahci-test: skip when the scratch files cannot be created
  tests/qtest/hd-geo-test: skip when the scratch file cannot be created
  tests/qtest/libqtest: create images with a byte-precise size
  tests/qtest/ide-test: build the shared disks with qemu-img
  tests/qtest/hd-geo-test: build the test images with qemu-img

 tests/qtest/ahci-test.c        |  48 ++++++++---
 tests/qtest/hd-geo-test.c      | 145 ++++++++++++++++++++++-----------
 tests/qtest/ide-test.c         |  96 +++++++++++++++++-----
 tests/qtest/libqos/libqos.c    |   4 +-
 tests/qtest/libqos/libqos.h    |   2 +-
 tests/qtest/libqtest.c         |   9 +-
 tests/qtest/libqtest.h         |  12 +++
 tests/qtest/virtio-scsi-test.c |   5 +-
 8 files changed, 237 insertions(+), 84 deletions(-)

-- 
2.53.0
Re: [PATCH 0/7] tests/qtest: fix the disk tests on a host with a small /tmp
Posted by Thomas Huth 1 month ago
  Hi Denis!

Please make sure to CC: the qtest maintainer (Fabiano Rosas) on series like 
this.

On 24/08/2026 22.06, Denis V. Lunev wrote:
> John reported ide-test dying at startup in the NetBSD VM:
> 
>    ERROR:../src/tests/qtest/ide-test.c:1260:main: assertion failed:
>    (ret == 0)
> 
> The hypothesys is that the ftruncate() of a 64 MiB scratch image. NetBSD
> mounts /tmp as a tmpfs sized at 25% of RAM and charges a file its full
> length themoment it is extended, so a sparse image is not free there and
> the call returns ENOSPC. The assert dates to 2013, nothing regressed.

Does NetBSD have another file like /var/tmp that might be friendlier to 
sparse files? If so, maybe that should be used instead?

OTOH, our NetBSD VM in tests/vm/ uses 4G of RAM, so a temporary file with 
just 64 MiB should really not be a problem...?

...> Hope this should help. John, can you give a try to this in the test?
> I do not have NetBSD handy and will start with this tomorrow if you
> will share guest config.
Simply type "make vm-build-netbsd" from your build folder, this should take 
care of everything automatically. No need to install NetBSD on your own.

  Thomas
Re: [PATCH 0/7] tests/qtest: fix the disk tests on a host with a small /tmp
Posted by Daniel P. Berrangé 1 month ago
On Thu, Aug 27, 2026 at 12:16:56PM +0200, Thomas Huth wrote:
> 
>  Hi Denis!
> 
> Please make sure to CC: the qtest maintainer (Fabiano Rosas) on series like
> this.
> 
> On 24/08/2026 22.06, Denis V. Lunev wrote:
> > John reported ide-test dying at startup in the NetBSD VM:
> > 
> >    ERROR:../src/tests/qtest/ide-test.c:1260:main: assertion failed:
> >    (ret == 0)
> > 
> > The hypothesys is that the ftruncate() of a 64 MiB scratch image. NetBSD
> > mounts /tmp as a tmpfs sized at 25% of RAM and charges a file its full
> > length themoment it is extended, so a sparse image is not free there and
> > the call returns ENOSPC. The assert dates to 2013, nothing regressed.
> 
> Does NetBSD have another file like /var/tmp that might be friendlier to
> sparse files? If so, maybe that should be used instead?
> 
> OTOH, our NetBSD VM in tests/vm/ uses 4G of RAM, so a temporary file with
> just 64 MiB should really not be a problem...?

Or this is a concurrency scaling problem, or racing with another test
that also uses stuff ?

We hard code memory to 4 GB, but -smp we scale to "$NUM-CPUs / 2"
for the "make vm-build-DIST" commands.

IOW, regardless of whether QEMU is launched with -smp 1 or -smp 20,
we only give it 4 GB to play with. We could be using a lot of RAM
for concurrent build jobs leaving almost nothing for the tmpfs for
the test.

If we were that close to exhaustion I'd expected to see out of
memory errors, but I'm unclear what NetBSD's behaviour is in
this respect ?  Maybe normal RAM usage can be pushed to swap
(of which I see another 4 GB) while tmpfs can't be pushed
to swap ?




With regards,
Daniel
-- 
|: https://berrange.com       ~~        https://hachyderm.io/@berrange :|
|: https://libvirt.org          ~~          https://entangle-photo.org :|
|: https://pixelfed.art/berrange   ~~    https://fstop138.berrange.com :|
Re: [PATCH 0/7] tests/qtest: fix the disk tests on a host with a small /tmp
Posted by Denis V. Lunev 3 weeks, 4 days ago
On 8/27/26 13:03, Daniel P. Berrangé wrote:
> On Thu, Aug 27, 2026 at 12:16:56PM +0200, Thomas Huth wrote:
>>  Hi Denis!
>>
>> Please make sure to CC: the qtest maintainer (Fabiano Rosas) on series like
>> this.
>>
>> On 24/08/2026 22.06, Denis V. Lunev wrote:
>>> John reported ide-test dying at startup in the NetBSD VM:
>>>
>>>    ERROR:../src/tests/qtest/ide-test.c:1260:main: assertion failed:
>>>    (ret == 0)
>>>
>>> The hypothesys is that the ftruncate() of a 64 MiB scratch image. NetBSD
>>> mounts /tmp as a tmpfs sized at 25% of RAM and charges a file its full
>>> length themoment it is extended, so a sparse image is not free there and
>>> the call returns ENOSPC. The assert dates to 2013, nothing regressed.
>> Does NetBSD have another file like /var/tmp that might be friendlier to
>> sparse files? If so, maybe that should be used instead?
>>
>> OTOH, our NetBSD VM in tests/vm/ uses 4G of RAM, so a temporary file with
>> just 64 MiB should really not be a problem...?
> Or this is a concurrency scaling problem, or racing with another test
> that also uses stuff ?
>
> We hard code memory to 4 GB, but -smp we scale to "$NUM-CPUs / 2"
> for the "make vm-build-DIST" commands.
>
> IOW, regardless of whether QEMU is launched with -smp 1 or -smp 20,
> we only give it 4 GB to play with. We could be using a lot of RAM
> for concurrent build jobs leaving almost nothing for the tmpfs for
> the test.
>
> If we were that close to exhaustion I'd expected to see out of
> memory errors, but I'm unclear what NetBSD's behaviour is in
> this respect ?  Maybe normal RAM usage can be pushed to swap
> (of which I see another 4 GB) while tmpfs can't be pushed
> to swap ?
>
>
>
>
> With regards,
> Daniel
Guys,

will somebody take a look into what was done?
This makes sense anyway - unified same class error tracking
plus requirements reduction.

Or if nobody care I could just push along with another
test fix?

Thank you in advance,
    Den

Re: [PATCH 0/7] tests/qtest: fix the disk tests on a host with a small /tmp
Posted by Fabiano Rosas 3 weeks, 2 days ago
"Denis V. Lunev" <den@virtuozzo.com> writes:

> On 8/27/26 13:03, Daniel P. Berrangé wrote:
>> On Thu, Aug 27, 2026 at 12:16:56PM +0200, Thomas Huth wrote:
>>>  Hi Denis!
>>>
>>> Please make sure to CC: the qtest maintainer (Fabiano Rosas) on series like
>>> this.
>>>
>>> On 24/08/2026 22.06, Denis V. Lunev wrote:
>>>> John reported ide-test dying at startup in the NetBSD VM:
>>>>
>>>>    ERROR:../src/tests/qtest/ide-test.c:1260:main: assertion failed:
>>>>    (ret == 0)
>>>>
>>>> The hypothesys is that the ftruncate() of a 64 MiB scratch image. NetBSD
>>>> mounts /tmp as a tmpfs sized at 25% of RAM and charges a file its full
>>>> length themoment it is extended, so a sparse image is not free there and
>>>> the call returns ENOSPC. The assert dates to 2013, nothing regressed.
>>> Does NetBSD have another file like /var/tmp that might be friendlier to
>>> sparse files? If so, maybe that should be used instead?
>>>
>>> OTOH, our NetBSD VM in tests/vm/ uses 4G of RAM, so a temporary file with
>>> just 64 MiB should really not be a problem...?
>> Or this is a concurrency scaling problem, or racing with another test
>> that also uses stuff ?
>>
>> We hard code memory to 4 GB, but -smp we scale to "$NUM-CPUs / 2"
>> for the "make vm-build-DIST" commands.
>>
>> IOW, regardless of whether QEMU is launched with -smp 1 or -smp 20,
>> we only give it 4 GB to play with. We could be using a lot of RAM
>> for concurrent build jobs leaving almost nothing for the tmpfs for
>> the test.
>>
>> If we were that close to exhaustion I'd expected to see out of
>> memory errors, but I'm unclear what NetBSD's behaviour is in
>> this respect ?  Maybe normal RAM usage can be pushed to swap
>> (of which I see another 4 GB) while tmpfs can't be pushed
>> to swap ?
>>
>>
>>
>>
>> With regards,
>> Daniel
> Guys,
>
> will somebody take a look into what was done?
> This makes sense anyway - unified same class error tracking
> plus requirements reduction.
>

I cannot reproduce this. I also don't think ENOSPC on 64MB is something
worth the churn at all.

> Or if nobody care I could just push along with another
> test fix?
>
> Thank you in advance,
>     Den
Re: [PATCH 0/7] tests/qtest: fix the disk tests on a host with a small /tmp
Posted by Denis V. Lunev 3 weeks, 2 days ago
On 9/3/26 18:57, Fabiano Rosas wrote:
> "Denis V. Lunev" <den@virtuozzo.com> writes:
>
>> On 8/27/26 13:03, Daniel P. Berrangé wrote:
>>> On Thu, Aug 27, 2026 at 12:16:56PM +0200, Thomas Huth wrote:
>>>>  Hi Denis!
>>>>
>>>> Please make sure to CC: the qtest maintainer (Fabiano Rosas) on series like
>>>> this.
>>>>
>>>> On 24/08/2026 22.06, Denis V. Lunev wrote:
>>>>> John reported ide-test dying at startup in the NetBSD VM:
>>>>>
>>>>>    ERROR:../src/tests/qtest/ide-test.c:1260:main: assertion failed:
>>>>>    (ret == 0)
>>>>>
>>>>> The hypothesys is that the ftruncate() of a 64 MiB scratch image. NetBSD
>>>>> mounts /tmp as a tmpfs sized at 25% of RAM and charges a file its full
>>>>> length themoment it is extended, so a sparse image is not free there and
>>>>> the call returns ENOSPC. The assert dates to 2013, nothing regressed.
>>>> Does NetBSD have another file like /var/tmp that might be friendlier to
>>>> sparse files? If so, maybe that should be used instead?
>>>>
>>>> OTOH, our NetBSD VM in tests/vm/ uses 4G of RAM, so a temporary file with
>>>> just 64 MiB should really not be a problem...?
>>> Or this is a concurrency scaling problem, or racing with another test
>>> that also uses stuff ?
>>>
>>> We hard code memory to 4 GB, but -smp we scale to "$NUM-CPUs / 2"
>>> for the "make vm-build-DIST" commands.
>>>
>>> IOW, regardless of whether QEMU is launched with -smp 1 or -smp 20,
>>> we only give it 4 GB to play with. We could be using a lot of RAM
>>> for concurrent build jobs leaving almost nothing for the tmpfs for
>>> the test.
>>>
>>> If we were that close to exhaustion I'd expected to see out of
>>> memory errors, but I'm unclear what NetBSD's behaviour is in
>>> this respect ?  Maybe normal RAM usage can be pushed to swap
>>> (of which I see another 4 GB) while tmpfs can't be pushed
>>> to swap ?
>>>
>>>
>>>
>>>
>>> With regards,
>>> Daniel
>> Guys,
>>
>> will somebody take a look into what was done?
>> This makes sense anyway - unified same class error tracking
>> plus requirements reduction.
>>
> I cannot reproduce this. I also don't think ENOSPC on 64MB is something
> worth the churn at all.
Great.

Clearly postponed, dropping this from watch list.

Thanks a lot for your time.

Den