Hello Jan, and Michal,
commit: 88957116c9 ("qemu: Use memory-backend-* for regular guest memory") gets
the system memory sharable without numa config.
The qemu options with the patch will be like as:
-machine pc-q35-5.2,accel=kvm,usb=off,vmport=off,smm=on,dump-guest-core=off,memory-backend=pc.ram \
-object memory-backend-file,id=pc.ram,mem-path=/var/lib/libvirt/qemu/ram/2-Test/pc.ram,share=yes,size=17179869184 \
So, we can remove the numa restriction of virtiofs, right?
The patch to remove that is the bottom of this email.
And, 88957116c9 seems to introduce another restriction which we
cannot create numa nodes on the machine. I got the following
message when I set the numa config and started the VM:
2020-10-02T00:31:46.780374Z qemu-system-x86_64: '-machine memory-backend' and '-numa memdev' properties are mutually exclusive
qemu rejects the -machine memory-backend' and '-numa memdev' because it
may cause crash. commit: ea81f98bce ("numa: prevent usage of -M memory-backend
and -numa memdev at the same time") introduced the check.
Do you have any ideas to solve above?
===================================================================
From: Masayoshi Mizuma <m.mizuma@jp.fujitsu.com>
Date: Fri, 2 Oct 2020 10:50:06 -0400
Subject: [PATCH] qemu: Remove the number of numa restriction for virtiofs
virtiofs requires at least one numa node to access the system
memory as shared memory.
88957116c9 ("qemu: Use memory-backend-* for regular guest memory")
gets the system memory sharable without the numa config.
Let's remove the number of numa restriction for virtiofs.
Signed-off-by: Masayoshi Mizuma <m.mizuma@jp.fujitsu.com>
---
src/qemu/qemu_validate.c | 44 ++++++++++++++++++++--------------------
1 file changed, 22 insertions(+), 22 deletions(-)
diff --git a/src/qemu/qemu_validate.c b/src/qemu/qemu_validate.c
index a212605579..b5372e7499 100644
--- a/src/qemu/qemu_validate.c
+++ b/src/qemu/qemu_validate.c
@@ -3475,36 +3475,36 @@ qemuValidateDomainDefVirtioFSSharedMemory(const virDomainDef *def)
size_t numa_nodes = virDomainNumaGetNodeCount(def->numa);
size_t i;
- if (numa_nodes == 0) {
+ if ((!numa_nodes) && (def->mem.access != VIR_DOMAIN_MEMORY_ACCESS_SHARED)) {
virReportError(VIR_ERR_CONFIG_UNSUPPORTED, "%s",
- _("virtiofs requires one or more NUMA nodes"));
+ _("virtiofs requires shared memory"));
return -1;
- }
-
- for (i = 0; i < numa_nodes; i++) {
- virDomainMemoryAccess node_access =
- virDomainNumaGetNodeMemoryAccessMode(def->numa, i);
+ } else {
+ for (i = 0; i < numa_nodes; i++) {
+ virDomainMemoryAccess node_access =
+ virDomainNumaGetNodeMemoryAccessMode(def->numa, i);
- switch (node_access) {
- case VIR_DOMAIN_MEMORY_ACCESS_DEFAULT:
- if (def->mem.access != VIR_DOMAIN_MEMORY_ACCESS_SHARED) {
+ switch (node_access) {
+ case VIR_DOMAIN_MEMORY_ACCESS_DEFAULT:
+ if (def->mem.access != VIR_DOMAIN_MEMORY_ACCESS_SHARED) {
+ virReportError(VIR_ERR_CONFIG_UNSUPPORTED, "%s",
+ _("virtiofs requires shared memory"));
+ return -1;
+ }
+ break;
+ case VIR_DOMAIN_MEMORY_ACCESS_SHARED:
+ break;
+ case VIR_DOMAIN_MEMORY_ACCESS_PRIVATE:
virReportError(VIR_ERR_CONFIG_UNSUPPORTED, "%s",
_("virtiofs requires shared memory"));
return -1;
- }
- break;
- case VIR_DOMAIN_MEMORY_ACCESS_SHARED:
- break;
- case VIR_DOMAIN_MEMORY_ACCESS_PRIVATE:
- virReportError(VIR_ERR_CONFIG_UNSUPPORTED, "%s",
- _("virtiofs requires shared memory"));
- return -1;
- case VIR_DOMAIN_MEMORY_ACCESS_LAST:
- default:
- virReportEnumRangeError(virDomainMemoryAccess, node_access);
- return -1;
+ case VIR_DOMAIN_MEMORY_ACCESS_LAST:
+ default:
+ virReportEnumRangeError(virDomainMemoryAccess, node_access);
+ return -1;
+ }
}
}
return 0;
--
2.27.0
Thanks,
Masa
On Fri, Oct 02, 2020 at 11:31:32AM -0400, Masayoshi Mizuma wrote:
> Hello Jan, and Michal,
>
> commit: 88957116c9 ("qemu: Use memory-backend-* for regular guest memory") gets
> the system memory sharable without numa config.
> The qemu options with the patch will be like as:
>
> -machine pc-q35-5.2,accel=kvm,usb=off,vmport=off,smm=on,dump-guest-core=off,memory-backend=pc.ram \
> -object memory-backend-file,id=pc.ram,mem-path=/var/lib/libvirt/qemu/ram/2-Test/pc.ram,share=yes,size=17179869184 \
>
> So, we can remove the numa restriction of virtiofs, right?
> The patch to remove that is the bottom of this email.
>
> And, 88957116c9 seems to introduce another restriction which we
> cannot create numa nodes on the machine. I got the following
> message when I set the numa config and started the VM:
>
> 2020-10-02T00:31:46.780374Z qemu-system-x86_64: '-machine memory-backend' and '-numa memdev' properties are mutually exclusive
It seems that this isn't a restriction for virtiofs. It's a bug introduced
by commit: 88957116c9. <numa> element doesn't work regardless of virtiofs config...
- Masa
>
> qemu rejects the -machine memory-backend' and '-numa memdev' because it
> may cause crash. commit: ea81f98bce ("numa: prevent usage of -M memory-backend
> and -numa memdev at the same time") introduced the check.
>
> Do you have any ideas to solve above?
>
> ===================================================================
> From: Masayoshi Mizuma <m.mizuma@jp.fujitsu.com>
> Date: Fri, 2 Oct 2020 10:50:06 -0400
> Subject: [PATCH] qemu: Remove the number of numa restriction for virtiofs
>
> virtiofs requires at least one numa node to access the system
> memory as shared memory.
>
> 88957116c9 ("qemu: Use memory-backend-* for regular guest memory")
> gets the system memory sharable without the numa config.
>
> Let's remove the number of numa restriction for virtiofs.
>
> Signed-off-by: Masayoshi Mizuma <m.mizuma@jp.fujitsu.com>
> ---
> src/qemu/qemu_validate.c | 44 ++++++++++++++++++++--------------------
> 1 file changed, 22 insertions(+), 22 deletions(-)
>
> diff --git a/src/qemu/qemu_validate.c b/src/qemu/qemu_validate.c
> index a212605579..b5372e7499 100644
> --- a/src/qemu/qemu_validate.c
> +++ b/src/qemu/qemu_validate.c
> @@ -3475,36 +3475,36 @@ qemuValidateDomainDefVirtioFSSharedMemory(const virDomainDef *def)
> size_t numa_nodes = virDomainNumaGetNodeCount(def->numa);
> size_t i;
>
> - if (numa_nodes == 0) {
> + if ((!numa_nodes) && (def->mem.access != VIR_DOMAIN_MEMORY_ACCESS_SHARED)) {
> virReportError(VIR_ERR_CONFIG_UNSUPPORTED, "%s",
> - _("virtiofs requires one or more NUMA nodes"));
> + _("virtiofs requires shared memory"));
> return -1;
> - }
> -
> - for (i = 0; i < numa_nodes; i++) {
> - virDomainMemoryAccess node_access =
> - virDomainNumaGetNodeMemoryAccessMode(def->numa, i);
> + } else {
> + for (i = 0; i < numa_nodes; i++) {
> + virDomainMemoryAccess node_access =
> + virDomainNumaGetNodeMemoryAccessMode(def->numa, i);
>
> - switch (node_access) {
> - case VIR_DOMAIN_MEMORY_ACCESS_DEFAULT:
> - if (def->mem.access != VIR_DOMAIN_MEMORY_ACCESS_SHARED) {
> + switch (node_access) {
> + case VIR_DOMAIN_MEMORY_ACCESS_DEFAULT:
> + if (def->mem.access != VIR_DOMAIN_MEMORY_ACCESS_SHARED) {
> + virReportError(VIR_ERR_CONFIG_UNSUPPORTED, "%s",
> + _("virtiofs requires shared memory"));
> + return -1;
> + }
> + break;
> + case VIR_DOMAIN_MEMORY_ACCESS_SHARED:
> + break;
> + case VIR_DOMAIN_MEMORY_ACCESS_PRIVATE:
> virReportError(VIR_ERR_CONFIG_UNSUPPORTED, "%s",
> _("virtiofs requires shared memory"));
> return -1;
> - }
> - break;
> - case VIR_DOMAIN_MEMORY_ACCESS_SHARED:
> - break;
> - case VIR_DOMAIN_MEMORY_ACCESS_PRIVATE:
> - virReportError(VIR_ERR_CONFIG_UNSUPPORTED, "%s",
> - _("virtiofs requires shared memory"));
> - return -1;
>
> - case VIR_DOMAIN_MEMORY_ACCESS_LAST:
> - default:
> - virReportEnumRangeError(virDomainMemoryAccess, node_access);
> - return -1;
> + case VIR_DOMAIN_MEMORY_ACCESS_LAST:
> + default:
> + virReportEnumRangeError(virDomainMemoryAccess, node_access);
> + return -1;
>
> + }
> }
> }
> return 0;
> --
> 2.27.0
>
> Thanks,
> Masa
>
On 10/3/20 1:15 AM, Masayoshi Mizuma wrote:
> On Fri, Oct 02, 2020 at 11:31:32AM -0400, Masayoshi Mizuma wrote:
>> Hello Jan, and Michal,
>>
>> commit: 88957116c9 ("qemu: Use memory-backend-* for regular guest memory") gets
>> the system memory sharable without numa config.
>> The qemu options with the patch will be like as:
>>
>> -machine pc-q35-5.2,accel=kvm,usb=off,vmport=off,smm=on,dump-guest-core=off,memory-backend=pc.ram \
>> -object memory-backend-file,id=pc.ram,mem-path=/var/lib/libvirt/qemu/ram/2-Test/pc.ram,share=yes,size=17179869184 \
>>
>> So, we can remove the numa restriction of virtiofs, right?
>> The patch to remove that is the bottom of this email.
>>
>> And, 88957116c9 seems to introduce another restriction which we
>> cannot create numa nodes on the machine. I got the following
>> message when I set the numa config and started the VM:
>>
>> 2020-10-02T00:31:46.780374Z qemu-system-x86_64: '-machine memory-backend' and '-numa memdev' properties are mutually exclusive
>
> It seems that this isn't a restriction for virtiofs. It's a bug introduced
> by commit: 88957116c9. <numa> element doesn't work regardless of virtiofs config...
Do you have domain XML that is failing?
Michal
On Mon, Oct 05, 2020 at 10:50:08AM +0200, Michal Privoznik wrote:
> On 10/3/20 1:15 AM, Masayoshi Mizuma wrote:
> > On Fri, Oct 02, 2020 at 11:31:32AM -0400, Masayoshi Mizuma wrote:
> > > Hello Jan, and Michal,
> > >
> > > commit: 88957116c9 ("qemu: Use memory-backend-* for regular guest memory") gets
> > > the system memory sharable without numa config.
> > > The qemu options with the patch will be like as:
> > >
> > > -machine pc-q35-5.2,accel=kvm,usb=off,vmport=off,smm=on,dump-guest-core=off,memory-backend=pc.ram \
> > > -object memory-backend-file,id=pc.ram,mem-path=/var/lib/libvirt/qemu/ram/2-Test/pc.ram,share=yes,size=17179869184 \
> > >
> > > So, we can remove the numa restriction of virtiofs, right?
> > > The patch to remove that is the bottom of this email.
> > >
> > > And, 88957116c9 seems to introduce another restriction which we
> > > cannot create numa nodes on the machine. I got the following
> > > message when I set the numa config and started the VM:
> > >
> > > 2020-10-02T00:31:46.780374Z qemu-system-x86_64: '-machine memory-backend' and '-numa memdev' properties are mutually exclusive
> >
> > It seems that this isn't a restriction for virtiofs. It's a bug introduced
> > by commit: 88957116c9. <numa> element doesn't work regardless of virtiofs config...
>
> Do you have domain XML that is failing?
Yes, I got the issue while virsh start:
# virsh start Test
error: Failed to start domain Test
error: internal error: qemu unexpectedly closed the monitor: 2020-10-05T17:31:21.078134Z qemu-system-x86_64: '-machine memory-backend' and '-numa memdev' properties are mutually exclusive
#
Here is the domain XML. I'm using libvirt with the head commit: 7d77fdb90f,
and qemu with the head commit: 36d9c2883e.
=========
<domain type='kvm'>
<name>Test</name>
<uuid>12785e5c-542e-4896-87f3-2e068fb7107f</uuid>
<memory unit='KiB'>8388608</memory>
<currentMemory unit='KiB'>8388608</currentMemory>
<vcpu placement='static'>4</vcpu>
<os>
<type arch='x86_64' machine='pc-q35-5.2'>hvm</type>
</os>
<features>
<acpi/>
<apic/>
<vmport state='off'/>
<smm state='on'/>
<vmcoreinfo state='on'/>
</features>
<cpu mode='host-model' check='partial'>
<numa>
<cell id='0' cpus='0-3' memory='8388608' unit='KiB'/>
</numa>
</cpu>
<clock offset='utc'>
<timer name='rtc' tickpolicy='catchup'/>
<timer name='pit' tickpolicy='delay'/>
<timer name='hpet' present='no'/>
</clock>
<on_poweroff>destroy</on_poweroff>
<on_reboot>restart</on_reboot>
<on_crash>destroy</on_crash>
<devices>
<emulator>/usr/local/bin/qemu-system-x86_64</emulator>
<disk type='file' device='disk'>
<driver name='qemu' type='qcow2'/>
<source file='/var/lib/libvirt/images/Test.qcow2'/>
<target dev='vda' bus='virtio'/>
<boot order='1'/>
<address type='pci' domain='0x0000' bus='0x04' slot='0x00' function='0x0'/>
</disk>
<controller type='pci' index='0' model='pcie-root'/>
<controller type='pci' index='1' model='pcie-root-port'>
<model name='pcie-root-port'/>
<target chassis='1' port='0x10'/>
<address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x0' multifunction='on'/>
</controller>
<controller type='pci' index='2' model='pcie-root-port'>
<model name='pcie-root-port'/>
<target chassis='2' port='0x9'/>
<address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x1'/>
</controller>
<controller type='pci' index='3' model='pcie-root-port'>
<model name='pcie-root-port'/>
<target chassis='3' port='0xa'/>
<address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x2'/>
</controller>
<controller type='pci' index='4' model='pcie-root-port'>
<model name='pcie-root-port'/>
<target chassis='4' port='0xb'/>
<address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x3'/>
</controller>
<controller type='pci' index='5' model='pcie-root-port'>
<model name='pcie-root-port'/>
<target chassis='5' port='0xc'/>
<address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x4'/>
</controller>
<controller type='pci' index='6' model='pcie-root-port'>
<model name='pcie-root-port'/>
<target chassis='6' port='0xd'/>
<address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x5'/>
</controller>
<controller type='usb' index='0' model='qemu-xhci'>
<address type='pci' domain='0x0000' bus='0x02' slot='0x00' function='0x0'/>
</controller>
<controller type='sata' index='0'>
<address type='pci' domain='0x0000' bus='0x00' slot='0x1f' function='0x2'/>
</controller>
<controller type='virtio-serial' index='0'>
<address type='pci' domain='0x0000' bus='0x03' slot='0x00' function='0x0'/>
</controller>
<interface type='network'>
<mac address='52:54:00:c1:a2:80'/>
<source network='default'/>
<model type='virtio'/>
<address type='pci' domain='0x0000' bus='0x01' slot='0x00' function='0x0'/>
</interface>
<serial type='pty'>
<target type='isa-serial' port='0'>
<model name='isa-serial'/>
</target>
</serial>
<console type='pty'>
<target type='serial' port='0'/>
</console>
<channel type='unix'>
<target type='virtio' name='org.qemu.guest_agent.0'/>
<address type='virtio-serial' controller='0' bus='0' port='1'/>
</channel>
<input type='mouse' bus='ps2'/>
<input type='keyboard' bus='ps2'/>
<memballoon model='virtio'>
<address type='pci' domain='0x0000' bus='0x05' slot='0x00' function='0x0'/>
</memballoon>
<rng model='virtio'>
<backend model='random'>/dev/urandom</backend>
<address type='pci' domain='0x0000' bus='0x06' slot='0x00' function='0x0'/>
</rng>
</devices>
</domain>
=========
Thanks,
Masa
© 2016 - 2024 Red Hat, Inc.