docs/formatdomain.html.in | 48 ++- docs/news.xml | 9 + docs/schemas/domaincommon.rng | 26 ++ po/POTFILES.in | 1 + src/Makefile.am | 1 + src/conf/device_conf.h | 1 + src/conf/domain_conf.c | 203 ++++++++++-- src/conf/domain_conf.h | 9 + src/libvirt_private.syms | 20 ++ src/qemu/qemu_command.c | 49 +++ src/qemu/qemu_command.h | 5 + src/qemu/qemu_domain.c | 23 +- src/qemu/qemu_domain.h | 1 + src/qemu/qemu_domain_address.c | 16 +- src/qemu/qemu_hostdev.c | 37 +++ src/qemu/qemu_hostdev.h | 8 + src/qemu/qemu_hotplug.c | 2 + src/security/security_apparmor.c | 3 + src/security/security_dac.c | 55 ++++ src/security/security_selinux.c | 54 ++++ src/util/virhostdev.c | 229 ++++++++++++- src/util/virhostdev.h | 16 + src/util/virmdev.c | 358 +++++++++++++++++++++ src/util/virmdev.h | 93 ++++++ tests/domaincapsschemadata/full.xml | 1 + .../qemuxml2argv-hostdev-mdev-unmanaged.args | 25 ++ .../qemuxml2argv-hostdev-mdev-unmanaged.xml | 37 +++ tests/qemuxml2argvtest.c | 6 + .../qemuxml2xmlout-hostdev-mdev-unmanaged.xml | 40 +++ tests/qemuxml2xmltest.c | 1 + 30 files changed, 1333 insertions(+), 44 deletions(-) create mode 100644 src/util/virmdev.c create mode 100644 src/util/virmdev.h create mode 100644 tests/qemuxml2argvdata/qemuxml2argv-hostdev-mdev-unmanaged.args create mode 100644 tests/qemuxml2argvdata/qemuxml2argv-hostdev-mdev-unmanaged.xml create mode 100644 tests/qemuxml2xmloutdata/qemuxml2xmlout-hostdev-mdev-unmanaged.xml
since the original v2 [1]: - resolved a few merge conflicts caused by @9d92f533 which refactored out some duplicate code which eventually lead to dropping patch 14/18 from the original series due to being unnecessary - rebased onto fresh HEAD [1] https://www.redhat.com/archives/libvir-list/2017-February/msg00739.html Erik Skultety (18): util: Introduce new module virmdev conf: Introduce new hostdev device type mdev conf: Introduce new address type mdev conf: Update XML parser, formatter, and RNG schema to support mdev conf: Introduce virDomainHostdevDefPostParse conf: Add post parse code for mdevs to virDomainHostdevDefPostParse security: dac: Enable labeling of vfio mediated devices security: selinux: Enable labeling of vfio mediated devices conf: Enable cold-plug of a mediated device qemu: Assign PCI addresses for mediated devices as well hostdev: Maintain a driver list of active mediated devices hostdev: Introduce a reattach method for mediated devices qemu: cgroup: Adjust cgroups' logic to allow mediated devices qemu: Bump the memory locking limit for mdevs as well qemu: Format mdevs on qemu command line test: Add some test cases for our test suite regarding the mdevs docs: Document the new hostdev and address type 'mdev' news: Update the NEWS.xml about the new mdev feature docs/formatdomain.html.in | 48 ++- docs/news.xml | 9 + docs/schemas/domaincommon.rng | 26 ++ po/POTFILES.in | 1 + src/Makefile.am | 1 + src/conf/device_conf.h | 1 + src/conf/domain_conf.c | 203 ++++++++++-- src/conf/domain_conf.h | 9 + src/libvirt_private.syms | 20 ++ src/qemu/qemu_command.c | 49 +++ src/qemu/qemu_command.h | 5 + src/qemu/qemu_domain.c | 23 +- src/qemu/qemu_domain.h | 1 + src/qemu/qemu_domain_address.c | 16 +- src/qemu/qemu_hostdev.c | 37 +++ src/qemu/qemu_hostdev.h | 8 + src/qemu/qemu_hotplug.c | 2 + src/security/security_apparmor.c | 3 + src/security/security_dac.c | 55 ++++ src/security/security_selinux.c | 54 ++++ src/util/virhostdev.c | 229 ++++++++++++- src/util/virhostdev.h | 16 + src/util/virmdev.c | 358 +++++++++++++++++++++ src/util/virmdev.h | 93 ++++++ tests/domaincapsschemadata/full.xml | 1 + .../qemuxml2argv-hostdev-mdev-unmanaged.args | 25 ++ .../qemuxml2argv-hostdev-mdev-unmanaged.xml | 37 +++ tests/qemuxml2argvtest.c | 6 + .../qemuxml2xmlout-hostdev-mdev-unmanaged.xml | 40 +++ tests/qemuxml2xmltest.c | 1 + 30 files changed, 1333 insertions(+), 44 deletions(-) create mode 100644 src/util/virmdev.c create mode 100644 src/util/virmdev.h create mode 100644 tests/qemuxml2argvdata/qemuxml2argv-hostdev-mdev-unmanaged.args create mode 100644 tests/qemuxml2argvdata/qemuxml2argv-hostdev-mdev-unmanaged.xml create mode 100644 tests/qemuxml2xmloutdata/qemuxml2xmlout-hostdev-mdev-unmanaged.xml -- 2.10.2 -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
On Mon, 20 Feb 2017 15:28:13 +0100 Erik Skultety <eskultet@redhat.com> wrote: > since the original v2 [1]: > - resolved a few merge conflicts caused by @9d92f533 which refactored out some > duplicate code which eventually lead to dropping patch 14/18 from the original > series due to being unnecessary > - rebased onto fresh HEAD > > [1] https://www.redhat.com/archives/libvir-list/2017-February/msg00739.html > > Erik Skultety (18): > util: Introduce new module virmdev > conf: Introduce new hostdev device type mdev > conf: Introduce new address type mdev > conf: Update XML parser, formatter, and RNG schema to support mdev > conf: Introduce virDomainHostdevDefPostParse > conf: Add post parse code for mdevs to virDomainHostdevDefPostParse > security: dac: Enable labeling of vfio mediated devices > security: selinux: Enable labeling of vfio mediated devices > conf: Enable cold-plug of a mediated device > qemu: Assign PCI addresses for mediated devices as well > hostdev: Maintain a driver list of active mediated devices > hostdev: Introduce a reattach method for mediated devices > qemu: cgroup: Adjust cgroups' logic to allow mediated devices > qemu: Bump the memory locking limit for mdevs as well > qemu: Format mdevs on qemu command line > test: Add some test cases for our test suite regarding the mdevs > docs: Document the new hostdev and address type 'mdev' > news: Update the NEWS.xml about the new mdev feature > > docs/formatdomain.html.in | 48 ++- > docs/news.xml | 9 + > docs/schemas/domaincommon.rng | 26 ++ > po/POTFILES.in | 1 + > src/Makefile.am | 1 + > src/conf/device_conf.h | 1 + > src/conf/domain_conf.c | 203 ++++++++++-- > src/conf/domain_conf.h | 9 + > src/libvirt_private.syms | 20 ++ I don't understand how these get generated, so I won't suggest where they should be added, but a usb test fails for me without adding these to this syms file: +virMediatedDeviceModelTypeFromString; +virMediatedDeviceModelTypeToString; Thanks, Alex > src/qemu/qemu_command.c | 49 +++ > src/qemu/qemu_command.h | 5 + > src/qemu/qemu_domain.c | 23 +- > src/qemu/qemu_domain.h | 1 + > src/qemu/qemu_domain_address.c | 16 +- > src/qemu/qemu_hostdev.c | 37 +++ > src/qemu/qemu_hostdev.h | 8 + > src/qemu/qemu_hotplug.c | 2 + > src/security/security_apparmor.c | 3 + > src/security/security_dac.c | 55 ++++ > src/security/security_selinux.c | 54 ++++ > src/util/virhostdev.c | 229 ++++++++++++- > src/util/virhostdev.h | 16 + > src/util/virmdev.c | 358 +++++++++++++++++++++ > src/util/virmdev.h | 93 ++++++ > tests/domaincapsschemadata/full.xml | 1 + > .../qemuxml2argv-hostdev-mdev-unmanaged.args | 25 ++ > .../qemuxml2argv-hostdev-mdev-unmanaged.xml | 37 +++ > tests/qemuxml2argvtest.c | 6 + > .../qemuxml2xmlout-hostdev-mdev-unmanaged.xml | 40 +++ > tests/qemuxml2xmltest.c | 1 + > 30 files changed, 1333 insertions(+), 44 deletions(-) > create mode 100644 src/util/virmdev.c > create mode 100644 src/util/virmdev.h > create mode 100644 tests/qemuxml2argvdata/qemuxml2argv-hostdev-mdev-unmanaged.args > create mode 100644 tests/qemuxml2argvdata/qemuxml2argv-hostdev-mdev-unmanaged.xml > create mode 100644 tests/qemuxml2xmloutdata/qemuxml2xmlout-hostdev-mdev-unmanaged.xml > -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
On Fri, Feb 24, 2017 at 11:10:00AM -0700, Alex Williamson wrote: > On Mon, 20 Feb 2017 15:28:13 +0100 > Erik Skultety <eskultet@redhat.com> wrote: > > > since the original v2 [1]: > > - resolved a few merge conflicts caused by @9d92f533 which refactored out some > > duplicate code which eventually lead to dropping patch 14/18 from the original > > series due to being unnecessary > > - rebased onto fresh HEAD > > > > [1] https://www.redhat.com/archives/libvir-list/2017-February/msg00739.html > > > > Erik Skultety (18): > > util: Introduce new module virmdev > > conf: Introduce new hostdev device type mdev > > conf: Introduce new address type mdev > > conf: Update XML parser, formatter, and RNG schema to support mdev > > conf: Introduce virDomainHostdevDefPostParse > > conf: Add post parse code for mdevs to virDomainHostdevDefPostParse > > security: dac: Enable labeling of vfio mediated devices > > security: selinux: Enable labeling of vfio mediated devices > > conf: Enable cold-plug of a mediated device > > qemu: Assign PCI addresses for mediated devices as well > > hostdev: Maintain a driver list of active mediated devices > > hostdev: Introduce a reattach method for mediated devices > > qemu: cgroup: Adjust cgroups' logic to allow mediated devices > > qemu: Bump the memory locking limit for mdevs as well > > qemu: Format mdevs on qemu command line > > test: Add some test cases for our test suite regarding the mdevs > > docs: Document the new hostdev and address type 'mdev' > > news: Update the NEWS.xml about the new mdev feature > > > > docs/formatdomain.html.in | 48 ++- > > docs/news.xml | 9 + > > docs/schemas/domaincommon.rng | 26 ++ > > po/POTFILES.in | 1 + > > src/Makefile.am | 1 + > > src/conf/device_conf.h | 1 + > > src/conf/domain_conf.c | 203 ++++++++++-- > > src/conf/domain_conf.h | 9 + > > src/libvirt_private.syms | 20 ++ > > I don't understand how these get generated, so I won't suggest where > they should be added, but a usb test fails for me without adding > these to this syms file: > Hmm, weird, nothing fails for me, even rebased onto the current master. Anyhow, I checked that we indeed put these generated methods into the .syms file which I didn't know (never had a problem with that), but I'll fix it. Thanks, Erik -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
On Mon, 27 Feb 2017 10:55:26 +0100 Erik Skultety <eskultet@redhat.com> wrote: > On Fri, Feb 24, 2017 at 11:10:00AM -0700, Alex Williamson wrote: > > On Mon, 20 Feb 2017 15:28:13 +0100 > > Erik Skultety <eskultet@redhat.com> wrote: > > > > > since the original v2 [1]: > > > - resolved a few merge conflicts caused by @9d92f533 which refactored out some > > > duplicate code which eventually lead to dropping patch 14/18 from the original > > > series due to being unnecessary > > > - rebased onto fresh HEAD > > > > > > [1] https://www.redhat.com/archives/libvir-list/2017-February/msg00739.html > > > > > > Erik Skultety (18): > > > util: Introduce new module virmdev > > > conf: Introduce new hostdev device type mdev > > > conf: Introduce new address type mdev > > > conf: Update XML parser, formatter, and RNG schema to support mdev > > > conf: Introduce virDomainHostdevDefPostParse > > > conf: Add post parse code for mdevs to virDomainHostdevDefPostParse > > > security: dac: Enable labeling of vfio mediated devices > > > security: selinux: Enable labeling of vfio mediated devices > > > conf: Enable cold-plug of a mediated device > > > qemu: Assign PCI addresses for mediated devices as well > > > hostdev: Maintain a driver list of active mediated devices > > > hostdev: Introduce a reattach method for mediated devices > > > qemu: cgroup: Adjust cgroups' logic to allow mediated devices > > > qemu: Bump the memory locking limit for mdevs as well > > > qemu: Format mdevs on qemu command line > > > test: Add some test cases for our test suite regarding the mdevs > > > docs: Document the new hostdev and address type 'mdev' > > > news: Update the NEWS.xml about the new mdev feature > > > > > > docs/formatdomain.html.in | 48 ++- > > > docs/news.xml | 9 + > > > docs/schemas/domaincommon.rng | 26 ++ > > > po/POTFILES.in | 1 + > > > src/Makefile.am | 1 + > > > src/conf/device_conf.h | 1 + > > > src/conf/domain_conf.c | 203 ++++++++++-- > > > src/conf/domain_conf.h | 9 + > > > src/libvirt_private.syms | 20 ++ > > > > I don't understand how these get generated, so I won't suggest where > > they should be added, but a usb test fails for me without adding > > these to this syms file: > > > > Hmm, weird, nothing fails for me, even rebased onto the current master. > Anyhow, I checked that we indeed put these generated methods into the .syms > file which I didn't know (never had a problem with that), but I'll fix it. The error occurs for me on the make rpm target with the following: FAIL: virusbtest ================ /home/alwillia/rpmbuild/BUILD/libvirt-3.1.0/tests/.libs/lt-virusbtest: symbol lookup error: /home/alwillia/rpmbuild/BUILD/libvirt-3.1.0/tests/.libs/virusbmock.so: undefined symbol: virMediatedDeviceModelTypeFromString FAIL virusbtest (exit status: 127) I'm based on the v3.1.0-rc1 tag plus this series. Thanks, Alex -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
On behalf of Yongli, He who tested these series with KVMGT vGPU mdev.
Hi, Erik
Here is the libvirt testing result. in general speaking, it works well. while start libvirt-d and starting the vm, there are some call traces, i attached them in the very end of this mail. this mail will be the test-by content later.
Test env summary
==============
0. Test Bed hardware summary
cat /proc/cpuinfo
vendor_id : GenuineIntel
cpu family : 6
model : 78
model name : Intel(R) Core(TM) i5-6260U CPU @ 1.80GHz
stepping : 3
microcode : 0x84
physical id : 0
cpu cores : 2
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic
sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc art arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf tsc_known_freq pni pclmulqdq dtes64 monitor ds_cpl vmx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm 3dnowprefetch epb intel_pt tpr_shadow vnmi flexpriority ept vpid fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms invpcid mpx rdseed adx smap clflushopt xsaveopt xsavec xgetbv1 xsaves dtherm ida arat pln pts hwp hwp_notify hwp_act_window hwp_epp
bugs :
1. test steps and software version following this guide, https://github.com/01org/gvt-linux/wiki/GVTg_Setup_Guide#322-build-kernel-source
2. Host env
uname -a
Linux z-nuc-11 4.10.0-vgt #1 SMP Thu Mar 9 15:55:49 CST
2017 x86_64 x86_64 x86_64 GNU/Linux
cat /etc/lsb-release
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=14.04
DISTRIB_CODENAME=trusty
DISTRIB_DESCRIPTION="Ubuntu 14.04.4 LTS"
kernel build from source:
git describe
gvt-fixes-2017-01-25-1560-gcedbc34
qemu build from source:
qemu-system-x86_64 --version
QEMU emulator version 2.8.50 (v2.8.0-2059-geba44e9-dirty)
Copyright (c) 2003-2017 Fabrice Bellard and the QEMU Project developers
libvirt, use your branch https://github.com/eskultety/libvirt.git
git describe
v1.2.17-rc2-5530-g202402e
git branch
master
* mdev-v3
3. guest xml template, refer to attachment
4. related scripts, refer to attachment
myvirsh: source build libvirt wrapper
gdb-libvirtd: libvirtd start up wrapper
the screen call trace while start the virtlogd ==================================
ubuntu@z-nuc-11:~/vgpu-meta/libvirt-stage$ ./libvirt/gdb-libvirt &
***************************************************
start virtlogd
[2] 2005
ubuntu@z-nuc-11:~/vgpu-meta/libvirt-stage$
***************************************************
start libvirt-d
2017-03-09 19:04:57.211+0000: 2059: info : libvirt version: 3.1.0
2017-03-09 19:04:57.211+0000: 2059: info : hostname: z-nuc-11.maas
2017-03-09 19:04:57.211+0000: 2059: error : qemuMonitorOpenUnix:367 :
failed to connect to monitor socket: No such process
2017-03-09 19:04:57.213+0000: 2059: error :
virMediatedDeviceGetIOMMUGroupDev:153 : internal error: IOMMU group file /sys/bus/mdev/devices/894f3983-1a36-42b3-b52c-1024aca216be/iommu_group
is not a symlink
2017-03-09 19:04:57.213+0000: 2003: info : libvirt version: 3.1.0
2017-03-09 19:04:57.213+0000: 2003: info : hostname: z-nuc-11.maas
2017-03-09 19:04:57.213+0000: 2003: error : virNetSocketReadWire:1800 :
End of file while reading data: Input/output error
the screen call trace while start the VM (same for Ubuntu, Win10 etc) ======================================================
ubuntu@z-nuc-11:~/vgpu-meta/libvirt-stage$ myvirsh start vgpu-ubuntu
2017-03-09 19:06:50.483+0000: 2232: info : libvirt version: 3.1.0
2017-03-09 19:06:50.483+0000: 2232: info : hostname: z-nuc-11.maas
2017-03-09 19:06:50.483+0000: 2232: warning : qemuDomainObjTaint:4056 :
Domain id=1 name='vgpu-ubuntu' uuid=972b5e38-0437-11e7-8f97-d36dba74552d
is tainted: high-privileges
2017-03-09 19:06:50.819+0000: 2204: info : libvirt version: 3.1.0
2017-03-09 19:06:50.819+0000: 2232: warning : virDomainAuditHostdev:456
: Unexpected hostdev type while encoding audit message: 4
2017-03-09 19:06:50.819+0000: 2204: info : hostname: z-nuc-11.maas
2017-03-09 19:06:50.819+0000: 2204: error : virNetSocketReadWire:1800 :
End of file while reading data: Input/output error Domain vgpu-ubuntu started
Tested-by: Yongli, He <yongli.he@intel.com>
>-----Original Message-----
>From: libvir-list-bounces@redhat.com [mailto:libvir-list-bounces@redhat.com]
>On Behalf Of Erik Skultety
>Sent: Monday, February 20, 2017 10:28 PM
>To: libvir-list@redhat.com
>Cc: Erik Skultety <eskultet@redhat.com>
>Subject: [libvirt] [RFC PATCH v2 REBASE 00/18] Introduce vGPU mdev framework
>to libvirt
>
>since the original v2 [1]:
>- resolved a few merge conflicts caused by @9d92f533 which refactored out
>some duplicate code which eventually lead to dropping patch 14/18 from the
>original series due to being unnecessary
>- rebased onto fresh HEAD
>
>[1] https://www.redhat.com/archives/libvir-list/2017-February/msg00739.html
>
>Erik Skultety (18):
> util: Introduce new module virmdev
> conf: Introduce new hostdev device type mdev
> conf: Introduce new address type mdev
> conf: Update XML parser, formatter, and RNG schema to support mdev
> conf: Introduce virDomainHostdevDefPostParse
> conf: Add post parse code for mdevs to virDomainHostdevDefPostParse
> security: dac: Enable labeling of vfio mediated devices
> security: selinux: Enable labeling of vfio mediated devices
> conf: Enable cold-plug of a mediated device
> qemu: Assign PCI addresses for mediated devices as well
> hostdev: Maintain a driver list of active mediated devices
> hostdev: Introduce a reattach method for mediated devices
> qemu: cgroup: Adjust cgroups' logic to allow mediated devices
> qemu: Bump the memory locking limit for mdevs as well
> qemu: Format mdevs on qemu command line
> test: Add some test cases for our test suite regarding the mdevs
> docs: Document the new hostdev and address type 'mdev'
> news: Update the NEWS.xml about the new mdev feature
>
> docs/formatdomain.html.in | 48 ++-
> docs/news.xml | 9 +
> docs/schemas/domaincommon.rng | 26 ++
> po/POTFILES.in | 1 +
> src/Makefile.am | 1 +
> src/conf/device_conf.h | 1 +
> src/conf/domain_conf.c | 203 ++++++++++--
> src/conf/domain_conf.h | 9 +
> src/libvirt_private.syms | 20 ++
> src/qemu/qemu_command.c | 49 +++
> src/qemu/qemu_command.h | 5 +
> src/qemu/qemu_domain.c | 23 +-
> src/qemu/qemu_domain.h | 1 +
> src/qemu/qemu_domain_address.c | 16 +-
> src/qemu/qemu_hostdev.c | 37 +++
> src/qemu/qemu_hostdev.h | 8 +
> src/qemu/qemu_hotplug.c | 2 +
> src/security/security_apparmor.c | 3 +
> src/security/security_dac.c | 55 ++++
> src/security/security_selinux.c | 54 ++++
> src/util/virhostdev.c | 229 ++++++++++++-
> src/util/virhostdev.h | 16 +
> src/util/virmdev.c | 358 +++++++++++++++++++++
> src/util/virmdev.h | 93 ++++++
> tests/domaincapsschemadata/full.xml | 1 +
> .../qemuxml2argv-hostdev-mdev-unmanaged.args | 25 ++
> .../qemuxml2argv-hostdev-mdev-unmanaged.xml | 37 +++
> tests/qemuxml2argvtest.c | 6 +
> .../qemuxml2xmlout-hostdev-mdev-unmanaged.xml | 40 +++
> tests/qemuxml2xmltest.c | 1 +
> 30 files changed, 1333 insertions(+), 44 deletions(-) create mode 100644
>src/util/virmdev.c create mode 100644 src/util/virmdev.h create mode 100644
>tests/qemuxml2argvdata/qemuxml2argv-hostdev-mdev-unmanaged.args
> create mode 100644 tests/qemuxml2argvdata/qemuxml2argv-hostdev-mdev-
>unmanaged.xml
> create mode 100644 tests/qemuxml2xmloutdata/qemuxml2xmlout-hostdev-
>mdev-unmanaged.xml
>
>--
>2.10.2
>
>--
>libvir-list mailing list
>libvir-list@redhat.com
>https://www.redhat.com/mailman/listinfo/libvir-list
# ps -ef |grep lt-libvirtd |grep -v grep | awk '{print $2}' | xargs gdb -p
LIBVIRT_PID=`ps -ef |grep lt-libvirtd |grep -v grep | grep -v gdb-libvirt | awk '{print $2}'`
echo $LIBVIRT_PID
LIBVIRT_PID=`cut -d " " -f1 <<< $LIBVIRT_PID`
if [ x$LIBVIRT_PID != x"" ]; then
echo "=================================================="
echo "attach libvirt-d"
echo "sudo gdb -p $LIBVIRT_PID"
sudo gdb -p $LIBVIRT_PID
else
echo "***************************************************"
echo "start libvirt-d"
# sudo $STAGE_DIR/libvirt//daemon/libvirtd --listen --verbose -f $STAGE_DIR/libvirt/daemon/libvirtd.conf
sudo $STAGE_DIR/libvirt//daemon/libvirtd --listen -f $STAGE_DIR/libvirt/daemon/libvirtd.conf
fi
sudo $STAGE_DIR/libvirt/run $STAGE_DIR/libvirt/tools/virsh $@
--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list
On 03/16/2017 03:17 AM, Chen, Xiaoguang wrote: > the screen call trace while start the VM (same for Ubuntu, Win10 etc) ====================================================== > > ubuntu@z-nuc-11:~/vgpu-meta/libvirt-stage$ myvirsh start vgpu-ubuntu > 2017-03-09 19:06:50.483+0000: 2232: info : libvirt version: 3.1.0 > 2017-03-09 19:06:50.483+0000: 2232: info : hostname: z-nuc-11.maas > 2017-03-09 19:06:50.483+0000: 2232: warning : qemuDomainObjTaint:4056 : > Domain id=1 name='vgpu-ubuntu' uuid=972b5e38-0437-11e7-8f97-d36dba74552d > is tainted: high-privileges I haven't considered any of the rest of the log yet, but this caught my eye on a first pass - "high-privileges" means that you're running qemu as root, so your test is bypassing several issues that could cause vfio device assignment to fail on a "standard" system. It shouldn't be necessary to run qemu as root in order for device assignment to work. Is there some specific reason that you're doing it this way? (I'm guessing that you've set "user = root" in /etc/libvirt/qemu.conf) -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
>-----Original Message----- >From: sendmail [mailto:justsendmailnothingelse@gmail.com] On Behalf Of Laine >Stump >Sent: Thursday, March 16, 2017 10:01 PM >To: libvir-list@redhat.com >Cc: Chen, Xiaoguang <xiaoguang.chen@intel.com>; Erik Skultety ><eskultet@redhat.com>; He, Yongli <yongli.he@intel.com> >Subject: Re: [libvirt] [RFC PATCH v2 REBASE 00/18] Introduce vGPU mdev >framework to libvirt > >On 03/16/2017 03:17 AM, Chen, Xiaoguang wrote: >> the screen call trace while start the VM (same for Ubuntu, Win10 etc) >> ====================================================== >> >> ubuntu@z-nuc-11:~/vgpu-meta/libvirt-stage$ myvirsh start vgpu-ubuntu >> 2017-03-09 19:06:50.483+0000: 2232: info : libvirt version: 3.1.0 >> 2017-03-09 19:06:50.483+0000: 2232: info : hostname: z-nuc-11.maas >> 2017-03-09 19:06:50.483+0000: 2232: warning : qemuDomainObjTaint:4056 : >> Domain id=1 name='vgpu-ubuntu' >> uuid=972b5e38-0437-11e7-8f97-d36dba74552d >> is tainted: high-privileges > >I haven't considered any of the rest of the log yet, but this caught my eye on a >first pass - "high-privileges" means that you're running qemu as root, so your test >is bypassing several issues that could cause vfio device assignment to fail on a >"standard" system. What do you mean for 'cause vfio device assignment to fail on a standard system'? > It shouldn't be necessary to run qemu as root in order for >device assignment to work. Is there some specific reason that you're doing it this >way? (I'm guessing that you've set "user = root" in /etc/libvirt/qemu.conf) No. we will test the v3 using a non-root user. -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
On 03/17/2017 01:57 AM, Chen, Xiaoguang wrote: > > >> -----Original Message----- >> From: sendmail [mailto:justsendmailnothingelse@gmail.com] On Behalf Of Laine >> Stump >> Sent: Thursday, March 16, 2017 10:01 PM >> To: libvir-list@redhat.com >> Cc: Chen, Xiaoguang <xiaoguang.chen@intel.com>; Erik Skultety >> <eskultet@redhat.com>; He, Yongli <yongli.he@intel.com> >> Subject: Re: [libvirt] [RFC PATCH v2 REBASE 00/18] Introduce vGPU mdev >> framework to libvirt >> >> On 03/16/2017 03:17 AM, Chen, Xiaoguang wrote: >>> the screen call trace while start the VM (same for Ubuntu, Win10 etc) >>> ====================================================== >>> >>> ubuntu@z-nuc-11:~/vgpu-meta/libvirt-stage$ myvirsh start vgpu-ubuntu >>> 2017-03-09 19:06:50.483+0000: 2232: info : libvirt version: 3.1.0 >>> 2017-03-09 19:06:50.483+0000: 2232: info : hostname: z-nuc-11.maas >>> 2017-03-09 19:06:50.483+0000: 2232: warning : qemuDomainObjTaint:4056 : >>> Domain id=1 name='vgpu-ubuntu' >>> uuid=972b5e38-0437-11e7-8f97-d36dba74552d >>> is tainted: high-privileges >> >> I haven't considered any of the rest of the log yet, but this caught my eye on a >> first pass - "high-privileges" means that you're running qemu as root, so your test >> is bypassing several issues that could cause vfio device assignment to fail on a >> "standard" system. > What do you mean for 'cause vfio device assignment to fail on a standard system'? I mean that there are some device/file setup operations that libvirt should be performing in order to make vfio device assignment work properly in the case that the qemu process is unprivileged, and those are often the source of bugs; if you run your tests with a privileged qemu process, you're not testing any of the code that performs those operations. > >> It shouldn't be necessary to run qemu as root in order for >> device assignment to work. Is there some specific reason that you're doing it this >> way? (I'm guessing that you've set "user = root" in /etc/libvirt/qemu.conf) > No. we will test the v3 using a non-root user. > -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
> [2] 2005 > ubuntu@z-nuc-11:~/vgpu-meta/libvirt-stage$ > *************************************************** > start libvirt-d > 2017-03-09 19:04:57.211+0000: 2059: info : libvirt version: 3.1.0 > 2017-03-09 19:04:57.211+0000: 2059: info : hostname: z-nuc-11.maas > 2017-03-09 19:04:57.211+0000: 2059: error : qemuMonitorOpenUnix:367 : > failed to connect to monitor socket: No such process > 2017-03-09 19:04:57.213+0000: 2059: error : > virMediatedDeviceGetIOMMUGroupDev:153 : internal error: IOMMU group file /sys/bus/mdev/devices/894f3983-1a36-42b3-b52c-1024aca216be/iommu_group > is not a symlink When I saw this error message for the first time in the original thread, I got confused, since this just checks whether the symlink exists, if it doesn't, the vfio device probably also doesn't exist (but take this with a grain of salt, I haven't investigated that deep) and libvirt needs it to pass it onto qemu command line. I hit this issue once by accident in the past and at that time I didn't understand what caused it, but after a reboot it was gone. So seeing it here it caught my eye and I investigated it last week. What I found out was that it's caused by the vfio-mdev module not being loaded automatically as a dependency. I solved it by autoloading the module on system boot. So this is not a libvirt issue, but just for a reference, there is a BZ on this [1]. > 2017-03-09 19:04:57.213+0000: 2003: info : libvirt version: 3.1.0 > 2017-03-09 19:04:57.213+0000: 2003: info : hostname: z-nuc-11.maas > 2017-03-09 19:04:57.213+0000: 2003: error : virNetSocketReadWire:1800 : > End of file while reading data: Input/output error I suppose this corresponds to the problem above, do you hit this error if you work around the vfio-mdev module problem described above? > > the screen call trace while start the VM (same for Ubuntu, Win10 etc) ====================================================== > > ubuntu@z-nuc-11:~/vgpu-meta/libvirt-stage$ myvirsh start vgpu-ubuntu > 2017-03-09 19:06:50.483+0000: 2232: info : libvirt version: 3.1.0 > 2017-03-09 19:06:50.483+0000: 2232: info : hostname: z-nuc-11.maas > 2017-03-09 19:06:50.483+0000: 2232: warning : qemuDomainObjTaint:4056 : > Domain id=1 name='vgpu-ubuntu' uuid=972b5e38-0437-11e7-8f97-d36dba74552d > is tainted: high-privileges > 2017-03-09 19:06:50.819+0000: 2204: info : libvirt version: 3.1.0 > 2017-03-09 19:06:50.819+0000: 2232: warning : virDomainAuditHostdev:456 > : Unexpected hostdev type while encoding audit message: 4 This one's interesting, again, are you able to hit the error when you work around the missing vfio-mdev module? I'll have a look at this if you actually can hit the error, even if the XML is correct. I posted v3 of the series and also created a new branch 'mdev-next' on my github [2]. I dropped the attribute 'type' from the source address element, so follow the example in the updated docs. Thanks for giving it a try. Erik [1] https://bugzilla.redhat.com/show_bug.cgi?id=1420572 [2] https://github.com/eskultety/libvirt/commits/mdev-next -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
>-----Original Message----- >From: Erik Skultety [mailto:eskultet@redhat.com] >Sent: Thursday, March 16, 2017 10:41 PM >To: Chen, Xiaoguang <xiaoguang.chen@intel.com> >Cc: libvir-list@redhat.com; He, Yongli <yongli.he@intel.com> >Subject: Re: [libvirt] [RFC PATCH v2 REBASE 00/18] Introduce vGPU mdev >framework to libvirt > >> [2] 2005 >> ubuntu@z-nuc-11:~/vgpu-meta/libvirt-stage$ >> *************************************************** >> start libvirt-d >> 2017-03-09 19:04:57.211+0000: 2059: info : libvirt version: 3.1.0 >> 2017-03-09 19:04:57.211+0000: 2059: info : hostname: z-nuc-11.maas >> 2017-03-09 19:04:57.211+0000: 2059: error : qemuMonitorOpenUnix:367 : >> failed to connect to monitor socket: No such process >> 2017-03-09 19:04:57.213+0000: 2059: error : >> virMediatedDeviceGetIOMMUGroupDev:153 : internal error: IOMMU group >> file >> /sys/bus/mdev/devices/894f3983-1a36-42b3-b52c- >1024aca216be/iommu_group >> is not a symlink > >When I saw this error message for the first time in the original thread, I got >confused, since this just checks whether the symlink exists, if it doesn't, the vfio >device probably also doesn't exist (but take this with a grain of salt, I haven't >investigated that deep) and libvirt needs it to pass it onto qemu command line. I >hit this issue once by accident in the past and at that time I didn't understand >what caused it, but after a reboot it was gone. >So seeing it here it caught my eye and I investigated it last week. What I found >out was that it's caused by the vfio-mdev module not being loaded automatically >as a dependency. I solved it by autoloading the module on system boot. So this is >not a libvirt issue, but just for a reference, there is a BZ on this [1]. > >> 2017-03-09 19:04:57.213+0000: 2003: info : libvirt version: 3.1.0 >> 2017-03-09 19:04:57.213+0000: 2003: info : hostname: z-nuc-11.maas >> 2017-03-09 19:04:57.213+0000: 2003: error : virNetSocketReadWire:1800 : >> End of file while reading data: Input/output error > >I suppose this corresponds to the problem above, do you hit this error if you work >around the vfio-mdev module problem described above? Based on the KVMGT setup guide we should add kvmgt, vfio-iommu-type1 and vfio-mdev modules into initfamfs. So I don't think it is the same problem. But we will double confirm it later. Yong li is on vacation this week when he come back we will do the test again. > >> >> the screen call trace while start the VM (same for Ubuntu, Win10 etc) >> ====================================================== >> >> ubuntu@z-nuc-11:~/vgpu-meta/libvirt-stage$ myvirsh start vgpu-ubuntu >> 2017-03-09 19:06:50.483+0000: 2232: info : libvirt version: 3.1.0 >> 2017-03-09 19:06:50.483+0000: 2232: info : hostname: z-nuc-11.maas >> 2017-03-09 19:06:50.483+0000: 2232: warning : qemuDomainObjTaint:4056 : >> Domain id=1 name='vgpu-ubuntu' >> uuid=972b5e38-0437-11e7-8f97-d36dba74552d >> is tainted: high-privileges >> 2017-03-09 19:06:50.819+0000: 2204: info : libvirt version: 3.1.0 >> 2017-03-09 19:06:50.819+0000: 2232: warning : >> virDomainAuditHostdev:456 >> : Unexpected hostdev type while encoding audit message: 4 > >This one's interesting, again, are you able to hit the error when you work around >the missing vfio-mdev module? I'll have a look at this if you actually can hit the >error, even if the XML is correct. > >I posted v3 of the series and also created a new branch 'mdev-next' on my github >[2]. I dropped the attribute 'type' from the source address element, so follow the >example in the updated docs. Will test the v3 when yong li back from vacation. > >Thanks for giving it a try. >Erik > >[1] https://bugzilla.redhat.com/show_bug.cgi?id=1420572 >[2] https://github.com/eskultety/libvirt/commits/mdev-next -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
tested v3, on the mdev-next branch:
none-root:
======
1. hacker the privilege operations
sudo sh -c "ulimit -l 3074424832 && exec su $LOGNAME"
sudo chown ubuntu:ubuntu /dev/vfio/0
RFC: i don't know the correct way to do such thing while build it from
sources. updated me, thanks.
2. myvirsh -c qemu:///session
#define ./libvirt/vgpu-win10.xml
Domain vgpu-win10 defined from ./libvirt/vgpu-win10.xml
#start vgpu-win10
Domain vgpu-win10 started
3.ps aux | grep qemu
ubuntu 3262 141 12.3 2929432 2017172 ? SLl 23:58 0:51
/usr/bin/qemu-system-x86_64 -name guest=vgpu-win10,debug-threads=on -S
-object
secret,id=masterKey0,format=raw,file=/home/ubuntu/.config/libvirt/qemu/lib/domain-1-vgpu-win10/master-key.aes
-machine pc-i440fx-2.3,accel=kvm,usb=off,dump-guest-core=off -m 1908
-realtime mlock=off -smp 2,sockets=2,cores=1,threads=1 -uuid
916c5c36-0437-11e7-a23d-830ed1295d00 -no-user-config -nodefaults
-chardev
socket,id=charmonitor,path=/home/ubuntu/.config/libvirt/qemu/lib/domain-1-vgpu-win10/monitor.sock,server,nowait
-mon chardev=charmonitor,id=monitor,mode=control -rtc base=utc
-no-shutdown -boot strict=on -device
piix3-usb-uhci,id=usb,bus=pci.0,addr=0x1.0x2 -drive
file=/home/ubuntu/vgpu-meta/libvirt-stage/win10-64.qcow2,format=qcow2,if=none,id=drive-ide0-0-0,cache=none,aio=native
-device
ide-hd,bus=ide.0,unit=0,drive=drive-ide0-0-0,id=ide0-0-0,bootindex=1
-chardev pty,id=charserial0 -device
isa-serial,chardev=charserial0,id=serial0 -vnc 127.0.0.1:0 -device
cirrus-vga,id=video0,bus=pci.0,addr=0x2 -device
vfio-pci,sysfsdev=/sys/bus/mdev/devices/894f3983-1a36-42b3-b52c-1024aca216be,bus=pci.0,addr=0x4
-device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x3 -msg timestamp=on
ubuntu 3276 0.0 0.0 10468 2216 pts/0 S+ 23:59 0:00 grep
--color=auto qemu
rooted mode
========
start libvirt-d trace:
--------------------------
this trace shows occasionally while starting the libvirt-d, not every time.
2017-03-19 19:22:45.559+0000: 13104: info : libvirt version: 3.2.0
2017-03-19 19:22:45.559+0000: 13104: info : hostname: z-nuc-11.maas
2017-03-19 19:22:45.559+0000: 13104: error : qemuMonitorOpenUnix:367 :
failed to connect to monitor socket: No such process
2017-03-19 19:22:45.562+0000: 13000: info : libvirt version: 3.2.0
2017-03-19 19:22:45.562+0000: 13000: info : hostname: z-nuc-11.maas
2017-03-19 19:22:45.562+0000: 13000: error : virNetSocketReadWire:1800 :
End of file while reading data: Input/output error
start domian trace:
--------------------------
some time there is trace while starting the domain.
2017-03-19 19:22:50.912+0000: 13034: warning : qemuDomainObjTaint:4113 :
Domain id=3 name='vgpu-win10' uuid=916c5c36-0437-11e7-a23d-830ed1295d00
is tainted: high-privileges
2017-03-19 19:22:51.859+0000: 13000: error : virNetSocketReadWire:1800 :
End of file while reading data: Input/output error
2017-03-19 19:22:51.859+0000: 13034: warning : virDomainAuditHostdev:456
: Unexpected hostdev type while encoding audit message: 4
Domain vgpu-win10 started
NOTES:
there is no traces under none root mode, though i don't think the
trace is related to user privilege. fix me.
Regards
Yongli He
On 2017年03月16日 22:41, Erik Skultety wrote:
>> [2] 2005
>> ubuntu@z-nuc-11:~/vgpu-meta/libvirt-stage$
>> ***************************************************
>> start libvirt-d
>> 2017-03-09 19:04:57.211+0000: 2059: info : libvirt version: 3.1.0
>> 2017-03-09 19:04:57.211+0000: 2059: info : hostname: z-nuc-11.maas
>> 2017-03-09 19:04:57.211+0000: 2059: error : qemuMonitorOpenUnix:367 :
>> failed to connect to monitor socket: No such process
>> 2017-03-09 19:04:57.213+0000: 2059: error :
>> virMediatedDeviceGetIOMMUGroupDev:153 : internal error: IOMMU group file /sys/bus/mdev/devices/894f3983-1a36-42b3-b52c-1024aca216be/iommu_group
>> is not a symlink
> When I saw this error message for the first time in the original thread, I got
> confused, since this just checks whether the symlink exists, if it
> doesn't, the vfio device probably also doesn't exist (but take this with a
> grain of salt, I haven't investigated that deep) and libvirt needs it to pass
> it onto qemu command line. I hit this issue once by accident in the past and at
> that time I didn't understand what caused it, but after a reboot it was gone.
> So seeing it here it caught my eye and I investigated it last week. What I
> found out was that it's caused by the vfio-mdev module not being loaded
> automatically as a dependency. I solved it by autoloading the module on system
> boot. So this is not a libvirt issue, but just for a reference, there is a BZ
> on this [1].
>
>> 2017-03-09 19:04:57.213+0000: 2003: info : libvirt version: 3.1.0
>> 2017-03-09 19:04:57.213+0000: 2003: info : hostname: z-nuc-11.maas
>> 2017-03-09 19:04:57.213+0000: 2003: error : virNetSocketReadWire:1800 :
>> End of file while reading data: Input/output error
> I suppose this corresponds to the problem above, do you hit this error if you
> work around the vfio-mdev module problem described above?
>
>> the screen call trace while start the VM (same for Ubuntu, Win10 etc) ======================================================
>>
>> ubuntu@z-nuc-11:~/vgpu-meta/libvirt-stage$ myvirsh start vgpu-ubuntu
>> 2017-03-09 19:06:50.483+0000: 2232: info : libvirt version: 3.1.0
>> 2017-03-09 19:06:50.483+0000: 2232: info : hostname: z-nuc-11.maas
>> 2017-03-09 19:06:50.483+0000: 2232: warning : qemuDomainObjTaint:4056 :
>> Domain id=1 name='vgpu-ubuntu' uuid=972b5e38-0437-11e7-8f97-d36dba74552d
>> is tainted: high-privileges
>> 2017-03-09 19:06:50.819+0000: 2204: info : libvirt version: 3.1.0
>> 2017-03-09 19:06:50.819+0000: 2232: warning : virDomainAuditHostdev:456
>> : Unexpected hostdev type while encoding audit message: 4
> This one's interesting, again, are you able to hit the error when you work
> around the missing vfio-mdev module? I'll have a look at this if you actually
> can hit the error, even if the XML is correct.
>
> I posted v3 of the series and also created a new branch 'mdev-next' on my
> github [2]. I dropped the attribute 'type' from the source address element, so
> follow the example in the updated docs.
>
> Thanks for giving it a try.
> Erik
>
> [1] https://bugzilla.redhat.com/show_bug.cgi?id=1420572
> [2] https://github.com/eskultety/libvirt/commits/mdev-next
--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list
On Mon, Mar 20, 2017 at 03:27:19PM +0800, yonglihe wrote: > > tested v3, on the mdev-next branch: > > none-root: > ====== > 1. hacker the privilege operations > sudo sh -c "ulimit -l 3074424832 && exec su $LOGNAME" > sudo chown ubuntu:ubuntu /dev/vfio/0 > > RFC: i don't know the correct way to do such thing while build it from > sources. updated me, thanks. I'm not sure what you're referring to, so could you be more specific, so we can work from there and find the right solution? [...] > rooted mode > ======== > > > start libvirt-d trace: > -------------------------- > > this trace shows occasionally while starting the libvirt-d, not every time. > > 2017-03-19 19:22:45.559+0000: 13104: info : libvirt version: 3.2.0 > 2017-03-19 19:22:45.559+0000: 13104: info : hostname: z-nuc-11.maas > 2017-03-19 19:22:45.559+0000: 13104: error : qemuMonitorOpenUnix:367 : > failed to connect to monitor socket: No such process > 2017-03-19 19:22:45.562+0000: 13000: info : libvirt version: 3.2.0 > 2017-03-19 19:22:45.562+0000: 13000: info : hostname: z-nuc-11.maas > 2017-03-19 19:22:45.562+0000: 13000: error : virNetSocketReadWire:1800 : End > of file while reading data: Input/output error Hmm, the virNetSocketReadWire might go away with the recent libvirt patches (I rebased my branch onto current master) if you're using either virtlogd (which is the default in qemu.conf on RHEL if I'm not mistaken) or virtlockd, so try with the rebased branch. But I doubt the qemuMonitorOpenUnix bit would disappear with the mentioned libvirt fix. But I will need a fresh and complete daemon log to investigate further, as well as libvirtd.conf and qemu.conf (just in case), so that I can compare to my environment, since I'm not able to reproduce this. [...] > NOTES: > there is no traces under none root mode, though i don't think the trace > is related to user privilege. fix me. It's indeed odd that you don't see the same behaviour when running qemu as root. As I said, I will need the daemon log and preferably the configs as well to investigate further. Thanks, Erik -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
On Mon, Mar 20, 2017 at 11:03:40AM +0100, Erik Skultety wrote: > On Mon, Mar 20, 2017 at 03:27:19PM +0800, yonglihe wrote: > > > > tested v3, on the mdev-next branch: > > > > none-root: > > ====== > > 1. hacker the privilege operations > > sudo sh -c "ulimit -l 3074424832 && exec su $LOGNAME" > > sudo chown ubuntu:ubuntu /dev/vfio/0 > > > > RFC: i don't know the correct way to do such thing while build it from > > sources. updated me, thanks. > > I'm not sure what you're referring to, so could you be more specific, so we can > work from there and find the right solution? > > [...] > > > rooted mode > > ======== > > > > > > start libvirt-d trace: > > -------------------------- > > > > this trace shows occasionally while starting the libvirt-d, not every time. > > > > 2017-03-19 19:22:45.559+0000: 13104: info : libvirt version: 3.2.0 > > 2017-03-19 19:22:45.559+0000: 13104: info : hostname: z-nuc-11.maas > > 2017-03-19 19:22:45.559+0000: 13104: error : qemuMonitorOpenUnix:367 : > > failed to connect to monitor socket: No such process > > 2017-03-19 19:22:45.562+0000: 13000: info : libvirt version: 3.2.0 > > 2017-03-19 19:22:45.562+0000: 13000: info : hostname: z-nuc-11.maas > > 2017-03-19 19:22:45.562+0000: 13000: error : virNetSocketReadWire:1800 : End > > of file while reading data: Input/output error > > Hmm, the virNetSocketReadWire might go away with the recent libvirt patches (I > rebased my branch onto current master) if you're using either virtlogd (which > is the default in qemu.conf on RHEL if I'm not mistaken) or virtlockd, so try ^^The fix I mentioned would only make difference if all the daemons would log into the same file or to journal, but the log format above is different from what you'd see in the journal and I assume there is a dedicated log file for each daemon, so please ignore my post above. I'll still need complete logs and the configs though. Erik > with the rebased branch. But I doubt the qemuMonitorOpenUnix bit would > disappear with the mentioned libvirt fix. But I will need a fresh and complete > daemon log to investigate further, as well as libvirtd.conf and qemu.conf (just > in case), so that I can compare to my environment, since I'm not able to > reproduce this. > > [...] > > > NOTES: > > there is no traces under none root mode, though i don't think the trace > > is related to user privilege. fix me. > > It's indeed odd that you don't see the same behaviour when running qemu as > root. As I said, I will need the daemon log and preferably the configs as well > to investigate further. > > Thanks, > Erik > > -- > libvir-list mailing list > libvir-list@redhat.com > https://www.redhat.com/mailman/listinfo/libvir-list -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
On 03/20/2017 07:37 PM, Erik Skultety wrote:
> On Mon, Mar 20, 2017 at 11:03:40AM +0100, Erik Skultety wrote:
>> On Mon, Mar 20, 2017 at 03:27:19PM +0800, yonglihe wrote:
>>> tested v3, on the mdev-next branch:
>>>
>>> none-root:
>>> ======
>>> 1. hacker the privilege operations
>>> sudo sh -c "ulimit -l 3074424832 && exec su $LOGNAME"
>>> sudo chown ubuntu:ubuntu /dev/vfio/0
>>>
>>> RFC: i don't know the correct way to do such thing while build it from
>>> sources. updated me, thanks.
>> I'm not sure what you're referring to, so could you be more specific, so we can
>> work from there and find the right solution?
while i try starting a non-privileged vm, there is 2 problem i run into:
1. the ulimit can not be changed cause the virsh user don't have that
right to do so.
2. the /dev/vfio/0 (i got only one mdev), is does not own by the user.
that's all I’m talking about.
>>
>> [...]
>>
>>> rooted mode
>>> ========
>>>
>>>
>>> start libvirt-d trace:
>>> --------------------------
>>>
>>> this trace shows occasionally while starting the libvirt-d, not every time.
>>>
>>> 2017-03-19 19:22:45.559+0000: 13104: info : libvirt version: 3.2.0
>>> 2017-03-19 19:22:45.559+0000: 13104: info : hostname: z-nuc-11.maas
>>> 2017-03-19 19:22:45.559+0000: 13104: error : qemuMonitorOpenUnix:367 :
>>> failed to connect to monitor socket: No such process
>>> 2017-03-19 19:22:45.562+0000: 13000: info : libvirt version: 3.2.0
>>> 2017-03-19 19:22:45.562+0000: 13000: info : hostname: z-nuc-11.maas
>>> 2017-03-19 19:22:45.562+0000: 13000: error : virNetSocketReadWire:1800 : End
>>> of file while reading data: Input/output error
>> Hmm, the virNetSocketReadWire might go away with the recent libvirt patches (I
>> rebased my branch onto current master) if you're using either virtlogd (which
>> is the default in qemu.conf on RHEL if I'm not mistaken) or virtlockd, so try
> ^^The fix I mentioned would only make difference if all the daemons would log
> into the same file or to journal, but the log format above is different from
> what you'd see in the journal and I assume there is a dedicated log file for
> each daemon, so please ignore my post above. I'll still need complete logs and
refer to attachment. the logfile is messed up. i will update you later.
i'm going to try pull new patches, and retest it.
Yongli He
> the configs though.
>
> Erik
>
>> with the rebased branch. But I doubt the qemuMonitorOpenUnix bit would
>> disappear with the mentioned libvirt fix. But I will need a fresh and complete
>> daemon log to investigate further, as well as libvirtd.conf and qemu.conf (just
>> in case), so that I can compare to my environment, since I'm not able to
>> reproduce this.
>>
>> [...]
>>
>>> NOTES:
>>> there is no traces under none root mode, though i don't think the trace
>>> is related to user privilege. fix me.
>> It's indeed odd that you don't see the same behaviour when running qemu as
>> root. As I said, I will need the daemon log and preferably the configs as well
>> to investigate further.
>>
>> Thanks,
>> Erik
>>
>> --
>> libvir-list mailing list
>> libvir-list@redhat.com
>> https://www.redhat.com/mailman/listinfo/libvir-list
# Master configuration file for the QEMU driver.
# All settings described here are optional - if omitted, sensible
# defaults are used.
# Use of TLS requires that x509 certificates be issued. The default is
# to keep them in /etc/pki/qemu. This directory must contain
#
# ca-cert.pem - the CA master certificate
# server-cert.pem - the server certificate signed with ca-cert.pem
# server-key.pem - the server private key
#
# and optionally may contain
#
# dh-params.pem - the DH params configuration file
#
#default_tls_x509_cert_dir = "/etc/pki/qemu"
# The default TLS configuration only uses certificates for the server
# allowing the client to verify the server's identity and establish
# an encrypted channel.
#
# It is possible to use x509 certificates for authentication too, by
# issuing a x509 certificate to every client who needs to connect.
#
# Enabling this option will reject any client who does not have a
# certificate signed by the CA in /etc/pki/qemu/ca-cert.pem
#
#default_tls_x509_verify = 1
#
# Libvirt assumes the server-key.pem file is unencrypted by default.
# To use an encrypted server-key.pem file, the password to decrypt
# the PEM file is required. This can be provided by creating a secret
# object in libvirt and then to uncomment this setting to set the UUID
# of the secret.
#
# NB This default all-zeros UUID will not work. Replace it with the
# output from the UUID for the TLS secret from a 'virsh secret-list'
# command and then uncomment the entry
#
#default_tls_x509_secret_uuid = "00000000-0000-0000-0000-000000000000"
# VNC is configured to listen on 127.0.0.1 by default.
# To make it listen on all public interfaces, uncomment
# this next option.
#
# NB, strong recommendation to enable TLS + x509 certificate
# verification when allowing public access
#
#vnc_listen = "0.0.0.0"
# Enable this option to have VNC served over an automatically created
# unix socket. This prevents unprivileged access from users on the
# host machine, though most VNC clients do not support it.
#
# This will only be enabled for VNC configurations that have listen
# type=address but without any address specified. This setting takes
# preference over vnc_listen.
#
#vnc_auto_unix_socket = 1
# Enable use of TLS encryption on the VNC server. This requires
# a VNC client which supports the VeNCrypt protocol extension.
# Examples include vinagre, virt-viewer, virt-manager and vencrypt
# itself. UltraVNC, RealVNC, TightVNC do not support this
#
# It is necessary to setup CA and issue a server certificate
# before enabling this.
#
#vnc_tls = 1
# In order to override the default TLS certificate location for
# vnc certificates, supply a valid path to the certificate directory.
# If the provided path does not exist then the default_tls_x509_cert_dir
# path will be used.
#
#vnc_tls_x509_cert_dir = "/etc/pki/libvirt-vnc"
# The default TLS configuration only uses certificates for the server
# allowing the client to verify the server's identity and establish
# an encrypted channel.
#
# It is possible to use x509 certificates for authentication too, by
# issuing a x509 certificate to every client who needs to connect.
#
# Enabling this option will reject any client who does not have a
# certificate signed by the CA in /etc/pki/libvirt-vnc/ca-cert.pem
#
# If this option is not supplied, it will be set to the value of
# "default_tls_x509_verify".
#
#vnc_tls_x509_verify = 1
# The default VNC password. Only 8 bytes are significant for
# VNC passwords. This parameter is only used if the per-domain
# XML config does not already provide a password. To allow
# access without passwords, leave this commented out. An empty
# string will still enable passwords, but be rejected by QEMU,
# effectively preventing any use of VNC. Obviously change this
# example here before you set this.
#
#vnc_password = "XYZ12345"
# Enable use of SASL encryption on the VNC server. This requires
# a VNC client which supports the SASL protocol extension.
# Examples include vinagre, virt-viewer and virt-manager
# itself. UltraVNC, RealVNC, TightVNC do not support this
#
# It is necessary to configure /etc/sasl2/qemu.conf to choose
# the desired SASL plugin (eg, GSSPI for Kerberos)
#
#vnc_sasl = 1
# The default SASL configuration file is located in /etc/sasl2/
# When running libvirtd unprivileged, it may be desirable to
# override the configs in this location. Set this parameter to
# point to the directory, and create a qemu.conf in that location
#
#vnc_sasl_dir = "/some/directory/sasl2"
# QEMU implements an extension for providing audio over a VNC connection,
# though if your VNC client does not support it, your only chance for getting
# sound output is through regular audio backends. By default, libvirt will
# disable all QEMU sound backends if using VNC, since they can cause
# permissions issues. Enabling this option will make libvirtd honor the
# QEMU_AUDIO_DRV environment variable when using VNC.
#
#vnc_allow_host_audio = 0
# SPICE is configured to listen on 127.0.0.1 by default.
# To make it listen on all public interfaces, uncomment
# this next option.
#
# NB, strong recommendation to enable TLS + x509 certificate
# verification when allowing public access
#
#spice_listen = "0.0.0.0"
# Enable use of TLS encryption on the SPICE server.
#
# It is necessary to setup CA and issue a server certificate
# before enabling this.
#
#spice_tls = 1
# In order to override the default TLS certificate location for
# spice certificates, supply a valid path to the certificate directory.
# If the provided path does not exist then the default_tls_x509_cert_dir
# path will be used.
#
#spice_tls_x509_cert_dir = "/etc/pki/libvirt-spice"
# Enable this option to have SPICE served over an automatically created
# unix socket. This prevents unprivileged access from users on the
# host machine.
#
# This will only be enabled for SPICE configurations that have listen
# type=address but without any address specified. This setting takes
# preference over spice_listen.
#
#spice_auto_unix_socket = 1
# The default SPICE password. This parameter is only used if the
# per-domain XML config does not already provide a password. To
# allow access without passwords, leave this commented out. An
# empty string will still enable passwords, but be rejected by
# QEMU, effectively preventing any use of SPICE. Obviously change
# this example here before you set this.
#
#spice_password = "XYZ12345"
# Enable use of SASL encryption on the SPICE server. This requires
# a SPICE client which supports the SASL protocol extension.
#
# It is necessary to configure /etc/sasl2/qemu.conf to choose
# the desired SASL plugin (eg, GSSPI for Kerberos)
#
#spice_sasl = 1
# The default SASL configuration file is located in /etc/sasl2/
# When running libvirtd unprivileged, it may be desirable to
# override the configs in this location. Set this parameter to
# point to the directory, and create a qemu.conf in that location
#
#spice_sasl_dir = "/some/directory/sasl2"
# Enable use of TLS encryption on the chardev TCP transports.
#
# It is necessary to setup CA and issue a server certificate
# before enabling this.
#
#chardev_tls = 1
# In order to override the default TLS certificate location for character
# device TCP certificates, supply a valid path to the certificate directory.
# If the provided path does not exist then the default_tls_x509_cert_dir
# path will be used.
#
#chardev_tls_x509_cert_dir = "/etc/pki/libvirt-chardev"
# The default TLS configuration only uses certificates for the server
# allowing the client to verify the server's identity and establish
# an encrypted channel.
#
# It is possible to use x509 certificates for authentication too, by
# issuing a x509 certificate to every client who needs to connect.
#
# Enabling this option will reject any client who does not have a
# certificate signed by the CA in /etc/pki/libvirt-chardev/ca-cert.pem
#
#chardev_tls_x509_verify = 1
# Uncomment and use the following option to override the default secret
# UUID provided in the default_tls_x509_secret_uuid parameter.
#
# NB This default all-zeros UUID will not work. Replace it with the
# output from the UUID for the TLS secret from a 'virsh secret-list'
# command and then uncomment the entry
#
#chardev_tls_x509_secret_uuid = "00000000-0000-0000-0000-000000000000"
# By default, if no graphical front end is configured, libvirt will disable
# QEMU audio output since directly talking to alsa/pulseaudio may not work
# with various security settings. If you know what you're doing, enable
# the setting below and libvirt will passthrough the QEMU_AUDIO_DRV
# environment variable when using nographics.
#
#nographics_allow_host_audio = 1
# Override the port for creating both VNC and SPICE sessions (min).
# This defaults to 5900 and increases for consecutive sessions
# or when ports are occupied, until it hits the maximum.
#
# Minimum must be greater than or equal to 5900 as lower number would
# result into negative vnc display number.
#
# Maximum must be less than 65536, because higher numbers do not make
# sense as a port number.
#
#remote_display_port_min = 5900
#remote_display_port_max = 65535
# VNC WebSocket port policies, same rules apply as with remote display
# ports. VNC WebSockets use similar display <-> port mappings, with
# the exception being that ports start from 5700 instead of 5900.
#
#remote_websocket_port_min = 5700
#remote_websocket_port_max = 65535
# The default security driver is SELinux. If SELinux is disabled
# on the host, then the security driver will automatically disable
# itself. If you wish to disable QEMU SELinux security driver while
# leaving SELinux enabled for the host in general, then set this
# to 'none' instead. It's also possible to use more than one security
# driver at the same time, for this use a list of names separated by
# comma and delimited by square brackets. For example:
#
# security_driver = [ "selinux", "apparmor" ]
#
# Notes: The DAC security driver is always enabled; as a result, the
# value of security_driver cannot contain "dac". The value "none" is
# a special value; security_driver can be set to that value in
# isolation, but it cannot appear in a list of drivers.
#
#security_driver = "selinux"
# If set to non-zero, then the default security labeling
# will make guests confined. If set to zero, then guests
# will be unconfined by default. Defaults to 1.
#security_default_confined = 1
# If set to non-zero, then attempts to create unconfined
# guests will be blocked. Defaults to 0.
#security_require_confined = 1
# The user for QEMU processes run by the system instance. It can be
# specified as a user name or as a user id. The qemu driver will try to
# parse this value first as a name and then, if the name doesn't exist,
# as a user id.
#
# Since a sequence of digits is a valid user name, a leading plus sign
# can be used to ensure that a user id will not be interpreted as a user
# name.
#
# Some examples of valid values are:
#
# user = "qemu" # A user named "qemu"
# user = "+0" # Super user (uid=0)
# user = "100" # A user named "100" or a user with uid=100
#
#user = "root"
# The group for QEMU processes run by the system instance. It can be
# specified in a similar way to user.
#group = "root"
# Whether libvirt should dynamically change file ownership
# to match the configured user/group above. Defaults to 1.
# Set to 0 to disable file ownership changes.
#dynamic_ownership = 1
# What cgroup controllers to make use of with QEMU guests
#
# - 'cpu' - use for schedular tunables
# - 'devices' - use for device whitelisting
# - 'memory' - use for memory tunables
# - 'blkio' - use for block devices I/O tunables
# - 'cpuset' - use for CPUs and memory nodes
# - 'cpuacct' - use for CPUs statistics.
#
# NB, even if configured here, they won't be used unless
# the administrator has mounted cgroups, e.g.:
#
# mkdir /dev/cgroup
# mount -t cgroup -o devices,cpu,memory,blkio,cpuset none /dev/cgroup
#
# They can be mounted anywhere, and different controllers
# can be mounted in different locations. libvirt will detect
# where they are located.
#
#cgroup_controllers = [ "cpu", "devices", "memory", "blkio", "cpuset", "cpuacct" ]
# This is the basic set of devices allowed / required by
# all virtual machines.
#
# As well as this, any configured block backed disks,
# all sound device, and all PTY devices are allowed.
#
# This will only need setting if newer QEMU suddenly
# wants some device we don't already know about.
#
#cgroup_device_acl = [
# "/dev/null", "/dev/full", "/dev/zero",
# "/dev/random", "/dev/urandom",
# "/dev/ptmx", "/dev/kvm", "/dev/kqemu",
# "/dev/rtc","/dev/hpet"
#]
#
# RDMA migration requires the following extra files to be added to the list:
# "/dev/infiniband/rdma_cm",
# "/dev/infiniband/issm0",
# "/dev/infiniband/issm1",
# "/dev/infiniband/umad0",
# "/dev/infiniband/umad1",
# "/dev/infiniband/uverbs0"
# The default format for QEMU/KVM guest save images is raw; that is, the
# memory from the domain is dumped out directly to a file. If you have
# guests with a large amount of memory, however, this can take up quite
# a bit of space. If you would like to compress the images while they
# are being saved to disk, you can also set "lzop", "gzip", "bzip2", or "xz"
# for save_image_format. Note that this means you slow down the process of
# saving a domain in order to save disk space; the list above is in descending
# order by performance and ascending order by compression ratio.
#
# save_image_format is used when you use 'virsh save' or 'virsh managedsave'
# at scheduled saving, and it is an error if the specified save_image_format
# is not valid, or the requested compression program can't be found.
#
# dump_image_format is used when you use 'virsh dump' at emergency
# crashdump, and if the specified dump_image_format is not valid, or
# the requested compression program can't be found, this falls
# back to "raw" compression.
#
# snapshot_image_format specifies the compression algorithm of the memory save
# image when an external snapshot of a domain is taken. This does not apply
# on disk image format. It is an error if the specified format isn't valid,
# or the requested compression program can't be found.
#
#save_image_format = "raw"
#dump_image_format = "raw"
#snapshot_image_format = "raw"
# When a domain is configured to be auto-dumped when libvirtd receives a
# watchdog event from qemu guest, libvirtd will save dump files in directory
# specified by auto_dump_path. Default value is /var/lib/libvirt/qemu/dump
#
#auto_dump_path = "/var/lib/libvirt/qemu/dump"
# When a domain is configured to be auto-dumped, enabling this flag
# has the same effect as using the VIR_DUMP_BYPASS_CACHE flag with the
# virDomainCoreDump API. That is, the system will avoid using the
# file system cache while writing the dump file, but may cause
# slower operation.
#
#auto_dump_bypass_cache = 0
# When a domain is configured to be auto-started, enabling this flag
# has the same effect as using the VIR_DOMAIN_START_BYPASS_CACHE flag
# with the virDomainCreateWithFlags API. That is, the system will
# avoid using the file system cache when restoring any managed state
# file, but may cause slower operation.
#
#auto_start_bypass_cache = 0
# If provided by the host and a hugetlbfs mount point is configured,
# a guest may request huge page backing. When this mount point is
# unspecified here, determination of a host mount point in /proc/mounts
# will be attempted. Specifying an explicit mount overrides detection
# of the same in /proc/mounts. Setting the mount point to "" will
# disable guest hugepage backing. If desired, multiple mount points can
# be specified at once, separated by comma and enclosed in square
# brackets, for example:
#
# hugetlbfs_mount = ["/dev/hugepages2M", "/dev/hugepages1G"]
#
# The size of huge page served by specific mount point is determined by
# libvirt at the daemon startup.
#
# NB, within these mount points, guests will create memory backing
# files in a location of $MOUNTPOINT/libvirt/qemu
#
#hugetlbfs_mount = "/dev/hugepages"
# Path to the setuid helper for creating tap devices. This executable
# is used to create <source type='bridge'> interfaces when libvirtd is
# running unprivileged. libvirt invokes the helper directly, instead
# of using "-netdev bridge", for security reasons.
#bridge_helper = "/usr/libexec/qemu-bridge-helper"
# If clear_emulator_capabilities is enabled, libvirt will drop all
# privileged capabilities of the QEmu/KVM emulator. This is enabled by
# default.
#
# Warning: Disabling this option means that a compromised guest can
# exploit the privileges and possibly do damage to the host.
#
#clear_emulator_capabilities = 1
# If enabled, libvirt will have QEMU set its process name to
# "qemu:VM_NAME", where VM_NAME is the name of the VM. The QEMU
# process will appear as "qemu:VM_NAME" in process listings and
# other system monitoring tools. By default, QEMU does not set
# its process title, so the complete QEMU command (emulator and
# its arguments) appear in process listings.
#
#set_process_name = 1
# If max_processes is set to a positive integer, libvirt will use
# it to set the maximum number of processes that can be run by qemu
# user. This can be used to override default value set by host OS.
# The same applies to max_files which sets the limit on the maximum
# number of opened files.
#
#max_processes = 0
#max_files = 0
# If max_core is set to a non-zero integer, then QEMU will be
# permitted to create core dumps when it crashes, provided its
# RAM size is smaller than the limit set.
#
# Be warned that the core dump will include a full copy of the
# guest RAM, if the 'dump_guest_core' setting has been enabled,
# or if the guest XML contains
#
# <memory dumpcore="on">...guest ram...</memory>
#
# If guest RAM is to be included, ensure the max_core limit
# is set to at least the size of the largest expected guest
# plus another 1GB for any QEMU host side memory mappings.
#
# As a special case it can be set to the string "unlimited" to
# to allow arbitrarily sized core dumps.
#
# By default the core dump size is set to 0 disabling all dumps
#
# Size is a positive integer specifying bytes or the
# string "unlimited"
#
#max_core = "unlimited"
# Determine if guest RAM is included in QEMU core dumps. By
# default guest RAM will be excluded if a new enough QEMU is
# present. Setting this to '1' will force guest RAM to always
# be included in QEMU core dumps.
#
# This setting will be ignored if the guest XML has set the
# dumpcore attribute on the <memory> element.
#
#dump_guest_core = 1
# mac_filter enables MAC addressed based filtering on bridge ports.
# This currently requires ebtables to be installed.
#
#mac_filter = 1
# By default, PCI devices below non-ACS switch are not allowed to be assigned
# to guests. By setting relaxed_acs_check to 1 such devices will be allowed to
# be assigned to guests.
#
#relaxed_acs_check = 1
# If allow_disk_format_probing is enabled, libvirt will probe disk
# images to attempt to identify their format, when not otherwise
# specified in the XML. This is disabled by default.
#
# WARNING: Enabling probing is a security hole in almost all
# deployments. It is strongly recommended that users update their
# guest XML <disk> elements to include <driver type='XXXX'/>
# elements instead of enabling this option.
#
#allow_disk_format_probing = 1
# In order to prevent accidentally starting two domains that
# share one writable disk, libvirt offers two approaches for
# locking files. The first one is sanlock, the other one,
# virtlockd, is then our own implementation. Accepted values
# are "sanlock" and "lockd".
#
#lock_manager = "lockd"
# Set limit of maximum APIs queued on one domain. All other APIs
# over this threshold will fail on acquiring job lock. Specially,
# setting to zero turns this feature off.
# Note, that job lock is per domain.
#
#max_queued = 0
###################################################################
# Keepalive protocol:
# This allows qemu driver to detect broken connections to remote
# libvirtd during peer-to-peer migration. A keepalive message is
# sent to the daemon after keepalive_interval seconds of inactivity
# to check if the daemon is still responding; keepalive_count is a
# maximum number of keepalive messages that are allowed to be sent
# to the daemon without getting any response before the connection
# is considered broken. In other words, the connection is
# automatically closed approximately after
# keepalive_interval * (keepalive_count + 1) seconds since the last
# message received from the daemon. If keepalive_interval is set to
# -1, qemu driver will not send keepalive requests during
# peer-to-peer migration; however, the remote libvirtd can still
# send them and source libvirtd will send responses. When
# keepalive_count is set to 0, connections will be automatically
# closed after keepalive_interval seconds of inactivity without
# sending any keepalive messages.
#
#keepalive_interval = 5
#keepalive_count = 5
# Use seccomp syscall whitelisting in QEMU.
# 1 = on, 0 = off, -1 = use QEMU default
# Defaults to -1.
#
#seccomp_sandbox = 1
# Override the listen address for all incoming migrations. Defaults to
# 0.0.0.0, or :: if both host and qemu are capable of IPv6.
#migration_address = "0.0.0.0"
# The default hostname or IP address which will be used by a migration
# source for transferring migration data to this host. The migration
# source has to be able to resolve this hostname and connect to it so
# setting "localhost" will not work. By default, the host's configured
# hostname is used.
#migration_host = "host.example.com"
# Override the port range used for incoming migrations.
#
# Minimum must be greater than 0, however when QEMU is not running as root,
# setting the minimum to be lower than 1024 will not work.
#
# Maximum must not be greater than 65535.
#
#migration_port_min = 49152
#migration_port_max = 49215
# Timestamp QEMU's log messages (if QEMU supports it)
#
# Defaults to 1.
#
#log_timestamp = 0
# Location of master nvram file
#
# When a domain is configured to use UEFI instead of standard
# BIOS it may use a separate storage for UEFI variables. If
# that's the case libvirt creates the variable store per domain
# using this master file as image. Each UEFI firmware can,
# however, have different variables store. Therefore the nvram is
# a list of strings when a single item is in form of:
# ${PATH_TO_UEFI_FW}:${PATH_TO_UEFI_VARS}.
# Later, when libvirt creates per domain variable store, this list is
# searched for the master image. The UEFI firmware can be called
# differently for different guest architectures. For instance, it's OVMF
# for x86_64 and i686, but it's AAVMF for aarch64. The libvirt default
# follows this scheme.
#nvram = [
# "/usr/share/OVMF/OVMF_CODE.fd:/usr/share/OVMF/OVMF_VARS.fd",
# "/usr/share/OVMF/OVMF_CODE.secboot.fd:/usr/share/OVMF/OVMF_VARS.fd",
# "/usr/share/AAVMF/AAVMF_CODE.fd:/usr/share/AAVMF/AAVMF_VARS.fd"
#]
# The backend to use for handling stdout/stderr output from
# QEMU processes.
#
# 'file': QEMU writes directly to a plain file. This is the
# historical default, but allows QEMU to inflict a
# denial of service attack on the host by exhausting
# filesystem space
#
# 'logd': QEMU writes to a pipe provided by virtlogd daemon.
# This is the current default, providing protection
# against denial of service by performing log file
# rollover when a size limit is hit.
#
#stdio_handler = "logd"
# QEMU gluster libgfapi log level, debug levels are 0-9, with 9 being the
# most verbose, and 0 representing no debugging output.
#
# The current logging levels defined in the gluster GFAPI are:
#
# 0 - None
# 1 - Emergency
# 2 - Alert
# 3 - Critical
# 4 - Error
# 5 - Warning
# 6 - Notice
# 7 - Info
# 8 - Debug
# 9 - Trace
#
# Defaults to 4
#
#gluster_debug_level = 9
# To enhance security, QEMU driver is capable of creating private namespaces
# for each domain started. Well, so far only "mount" namespace is supported. If
# enabled it means qemu process is unable to see all the devices on the system,
# only those configured for the domain in question. Libvirt then manages
# devices entries throughout the domain lifetime. This namespace is turned on
# by default.
#namespaces = [ "mount" ]
# This directory is used for memoryBacking source if configured as file.
# NOTE: big files will be stored here
#memory_backing_dir = "/var/lib/libvirt/qemu/ram"
# Master libvirt daemon configuration file
#
# For further information consult http://libvirt.org/format.html
#
# NOTE: the tests/daemon-conf regression test script requires
# that each "PARAMETER = VALUE" line in this file have the parameter
# name just after a leading "#".
#################################################################
#
# Network connectivity controls
#
# Flag listening for secure TLS connections on the public TCP/IP port.
# NB, must pass the --listen flag to the libvirtd process for this to
# have any effect.
#
# It is necessary to setup a CA and issue server certificates before
# using this capability.
#
# This is enabled by default, uncomment this to disable it
listen_tls = 0
# Listen for unencrypted TCP connections on the public TCP/IP port.
# NB, must pass the --listen flag to the libvirtd process for this to
# have any effect.
#
# Using the TCP socket requires SASL authentication by default. Only
# SASL mechanisms which support data encryption are allowed. This is
# DIGEST_MD5 and GSSAPI (Kerberos5)
#
# This is disabled by default, uncomment this to enable it.
#listen_tcp = 1
# Override the port for accepting secure TLS connections
# This can be a port number, or service name
#
#tls_port = "16514"
# Override the port for accepting insecure TCP connections
# This can be a port number, or service name
#
#tcp_port = "16509"
# Override the default configuration which binds to all network
# interfaces. This can be a numeric IPv4/6 address, or hostname
#
# If the libvirtd service is started in parallel with network
# startup (e.g. with systemd), binding to addresses other than
# the wildcards (0.0.0.0/::) might not be available yet.
#
#listen_addr = "192.168.0.1"
# Flag toggling mDNS advertizement of the libvirt service.
#
# Alternatively can disable for all services on a host by
# stopping the Avahi daemon
#
# This is disabled by default, uncomment this to enable it
#mdns_adv = 1
# Override the default mDNS advertizement name. This must be
# unique on the immediate broadcast network.
#
# The default is "Virtualization Host HOSTNAME", where HOSTNAME
# is substituted for the short hostname of the machine (without domain)
#
#mdns_name = "Virtualization Host Joe Demo"
#################################################################
#
# UNIX socket access controls
#
# Set the UNIX domain socket group ownership. This can be used to
# allow a 'trusted' set of users access to management capabilities
# without becoming root.
#
# This is restricted to 'root' by default.
#unix_sock_group = "libvirt"
# Set the UNIX socket permissions for the R/O socket. This is used
# for monitoring VM status only
#
# Default allows any user. If setting group ownership, you may want to
# restrict this too.
#unix_sock_ro_perms = "0777"
# Set the UNIX socket permissions for the R/W socket. This is used
# for full management of VMs
#
# Default allows only root. If PolicyKit is enabled on the socket,
# the default will change to allow everyone (eg, 0777)
#
# If not using PolicyKit and setting group ownership for access
# control, then you may want to relax this too.
#unix_sock_rw_perms = "0770"
# Set the UNIX socket permissions for the admin interface socket.
#
# Default allows only owner (root), do not change it unless you are
# sure to whom you are exposing the access to.
#unix_sock_admin_perms = "0700"
# Set the name of the directory in which sockets will be found/created.
#unix_sock_dir = "/var/run/libvirt"
#################################################################
#
# Authentication.
#
# - none: do not perform auth checks. If you can connect to the
# socket you are allowed. This is suitable if there are
# restrictions on connecting to the socket (eg, UNIX
# socket permissions), or if there is a lower layer in
# the network providing auth (eg, TLS/x509 certificates)
#
# - sasl: use SASL infrastructure. The actual auth scheme is then
# controlled from /etc/sasl2/libvirt.conf. For the TCP
# socket only GSSAPI & DIGEST-MD5 mechanisms will be used.
# For non-TCP or TLS sockets, any scheme is allowed.
#
# - polkit: use PolicyKit to authenticate. This is only suitable
# for use on the UNIX sockets. The default policy will
# require a user to supply their own password to gain
# full read/write access (aka sudo like), while anyone
# is allowed read/only access.
#
# Set an authentication scheme for UNIX read-only sockets
# By default socket permissions allow anyone to connect
#
# To restrict monitoring of domains you may wish to enable
# an authentication mechanism here
#auth_unix_ro = "none"
# Set an authentication scheme for UNIX read-write sockets
# By default socket permissions only allow root. If PolicyKit
# support was compiled into libvirt, the default will be to
# use 'polkit' auth.
#
# If the unix_sock_rw_perms are changed you may wish to enable
# an authentication mechanism here
#auth_unix_rw = "none"
# Change the authentication scheme for TCP sockets.
#
# If you don't enable SASL, then all TCP traffic is cleartext.
# Don't do this outside of a dev/test scenario. For real world
# use, always enable SASL and use the GSSAPI or DIGEST-MD5
# mechanism in /etc/sasl2/libvirt.conf
#auth_tcp = "sasl"
# Change the authentication scheme for TLS sockets.
#
# TLS sockets already have encryption provided by the TLS
# layer, and limited authentication is done by certificates
#
# It is possible to make use of any SASL authentication
# mechanism as well, by using 'sasl' for this option
#auth_tls = "none"
# Change the API access control scheme
#
# By default an authenticated user is allowed access
# to all APIs. Access drivers can place restrictions
# on this. By default the 'nop' driver is enabled,
# meaning no access control checks are done once a
# client has authenticated with libvirtd
#
#access_drivers = [ "polkit" ]
#################################################################
#
# TLS x509 certificate configuration
#
# Override the default server key file path
#
#key_file = "/etc/pki/libvirt/private/serverkey.pem"
# Override the default server certificate file path
#
#cert_file = "/etc/pki/libvirt/servercert.pem"
# Override the default CA certificate path
#
#ca_file = "/etc/pki/CA/cacert.pem"
# Specify a certificate revocation list.
#
# Defaults to not using a CRL, uncomment to enable it
#crl_file = "/etc/pki/CA/crl.pem"
#################################################################
#
# Authorization controls
#
# Flag to disable verification of our own server certificates
#
# When libvirtd starts it performs some sanity checks against
# its own certificates.
#
# Default is to always run sanity checks. Uncommenting this
# will disable sanity checks which is not a good idea
#tls_no_sanity_certificate = 1
# Flag to disable verification of client certificates
#
# Client certificate verification is the primary authentication mechanism.
# Any client which does not present a certificate signed by the CA
# will be rejected.
#
# Default is to always verify. Uncommenting this will disable
# verification - make sure an IP whitelist is set
#tls_no_verify_certificate = 1
# A whitelist of allowed x509 Distinguished Names
# This list may contain wildcards such as
#
# "C=GB,ST=London,L=London,O=Red Hat,CN=*"
#
# See the POSIX fnmatch function for the format of the wildcards.
#
# NB If this is an empty list, no client can connect, so comment out
# entirely rather than using empty list to disable these checks
#
# By default, no DN's are checked
#tls_allowed_dn_list = ["DN1", "DN2"]
# A whitelist of allowed SASL usernames. The format for username
# depends on the SASL authentication mechanism. Kerberos usernames
# look like username@REALM
#
# This list may contain wildcards such as
#
# "*@EXAMPLE.COM"
#
# See the POSIX fnmatch function for the format of the wildcards.
#
# NB If this is an empty list, no client can connect, so comment out
# entirely rather than using empty list to disable these checks
#
# By default, no Username's are checked
#sasl_allowed_username_list = ["joe@EXAMPLE.COM", "fred@EXAMPLE.COM" ]
# Override the compile time default TLS priority string. The
# default is usually "NORMAL" unless overridden at build time.
# Only set this is it is desired for libvirt to deviate from
# the global default settings.
#
#tls_priority="NORMAL"
#################################################################
#
# Processing controls
#
# The maximum number of concurrent client connections to allow
# over all sockets combined.
#max_clients = 5000
# The maximum length of queue of connections waiting to be
# accepted by the daemon. Note, that some protocols supporting
# retransmission may obey this so that a later reattempt at
# connection succeeds.
#max_queued_clients = 1000
# The maximum length of queue of accepted but not yet
# authenticated clients. The default value is 20. Set this to
# zero to turn this feature off.
#max_anonymous_clients = 20
# The minimum limit sets the number of workers to start up
# initially. If the number of active clients exceeds this,
# then more threads are spawned, up to max_workers limit.
# Typically you'd want max_workers to equal maximum number
# of clients allowed
#min_workers = 5
#max_workers = 20
# The number of priority workers. If all workers from above
# pool are stuck, some calls marked as high priority
# (notably domainDestroy) can be executed in this pool.
#prio_workers = 5
# Total global limit on concurrent RPC calls. Should be
# at least as large as max_workers. Beyond this, RPC requests
# will be read into memory and queued. This directly impacts
# memory usage, currently each request requires 256 KB of
# memory. So by default up to 5 MB of memory is used
#
# XXX this isn't actually enforced yet, only the per-client
# limit is used so far
#max_requests = 20
# Limit on concurrent requests from a single client
# connection. To avoid one client monopolizing the server
# this should be a small fraction of the global max_requests
# and max_workers parameter
#max_client_requests = 5
# Same processing controls, but this time for the admin interface.
# For description of each option, be so kind to scroll few lines
# upwards.
#admin_min_workers = 1
#admin_max_workers = 5
#admin_max_clients = 5
#admin_max_queued_clients = 5
#admin_max_client_requests = 5
#################################################################
#
# Logging controls
#
# Logging level: 4 errors, 3 warnings, 2 information, 1 debug
# basically 1 will log everything possible
# Note: Journald may employ rate limiting of the messages logged
# and thus lock up the libvirt daemon. To use the debug level with
# journald you have to specify it explicitly in 'log_outputs', otherwise
# only information level messages will be logged.
#log_level = 3
# Logging filters:
# A filter allows to select a different logging level for a given category
# of logs
# The format for a filter is one of:
# x:name
# x:+name
# where name is a string which is matched against the category
# given in the VIR_LOG_INIT() at the top of each libvirt source
# file, e.g., "remote", "qemu", or "util.json" (the name in the
# filter can be a substring of the full category name, in order
# to match multiple similar categories), the optional "+" prefix
# tells libvirt to log stack trace for each message matching
# name, and x is the minimal level where matching messages should
# be logged:
# 1: DEBUG
# 2: INFO
# 3: WARNING
# 4: ERROR
#
# Multiple filters can be defined in a single @filters, they just need to be
# separated by spaces.
#
# e.g. to only get warning or errors from the remote layer and only errors
# from the event layer:
#log_filters="3:remote 4:event"
# Logging outputs:
# An output is one of the places to save logging information
# The format for an output can be:
# x:stderr
# output goes to stderr
# x:syslog:name
# use syslog for the output and use the given name as the ident
# x:file:file_path
# output to a file, with the given filepath
# x:journald
# output to journald logging system
# In all case the x prefix is the minimal level, acting as a filter
# 1: DEBUG
# 2: INFO
# 3: WARNING
# 4: ERROR
#
# Multiple outputs can be defined, they just need to be separated by spaces.
# e.g. to log all warnings and errors to syslog under the libvirtd ident:
#log_outputs="3:syslog:libvirtd"
#
# Log debug buffer size:
#
# This configuration option is no longer used, since the global
# log buffer functionality has been removed. Please configure
# suitable log_outputs/log_filters settings to obtain logs.
#log_buffer_size = 64
##################################################################
#
# Auditing
#
# This setting allows usage of the auditing subsystem to be altered:
#
# audit_level == 0 -> disable all auditing
# audit_level == 1 -> enable auditing, only if enabled on host (default)
# audit_level == 2 -> enable auditing, and exit if disabled on host
#
#audit_level = 2
#
# If set to 1, then audit messages will also be sent
# via libvirt logging infrastructure. Defaults to 0
#
#audit_logging = 1
###################################################################
# UUID of the host:
# Host UUID is read from one of the sources specified in host_uuid_source.
#
# - 'smbios': fetch the UUID from 'dmidecode -s system-uuid'
# - 'machine-id': fetch the UUID from /etc/machine-id
#
# The host_uuid_source default is 'smbios'. If 'dmidecode' does not provide
# a valid UUID a temporary UUID will be generated.
#
# Another option is to specify host UUID in host_uuid.
#
# Keep the format of the example UUID below. UUID must not have all digits
# be the same.
# NB This default all-zeros UUID will not work. Replace
# it with the output of the 'uuidgen' command and then
# uncomment this entry
#host_uuid = "00000000-0000-0000-0000-000000000000"
#host_uuid_source = "smbios"
###################################################################
# Keepalive protocol:
# This allows libvirtd to detect broken client connections or even
# dead clients. A keepalive message is sent to a client after
# keepalive_interval seconds of inactivity to check if the client is
# still responding; keepalive_count is a maximum number of keepalive
# messages that are allowed to be sent to the client without getting
# any response before the connection is considered broken. In other
# words, the connection is automatically closed approximately after
# keepalive_interval * (keepalive_count + 1) seconds since the last
# message received from the client. If keepalive_interval is set to
# -1, libvirtd will never send keepalive requests; however clients
# can still send them and the daemon will send responses. When
# keepalive_count is set to 0, connections will be automatically
# closed after keepalive_interval seconds of inactivity without
# sending any keepalive messages.
#
#keepalive_interval = 5
#keepalive_count = 5
#
# These configuration options are no longer used. There is no way to
# restrict such clients from connecting since they first need to
# connect in order to ask for keepalive.
#
#keepalive_required = 1
#admin_keepalive_required = 1
# Keepalive settings for the admin interface
#admin_keepalive_interval = 5
#admin_keepalive_count = 5
###################################################################
# Open vSwitch:
# This allows to specify a timeout for openvswitch calls made by
# libvirt. The ovs-vsctl utility is used for the configuration and
# its timeout option is set by default to 5 seconds to avoid
# potential infinite waits blocking libvirt.
#
#ovs_timeout = 5
--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list
On 2017年03月16日 22:41, Erik Skultety wrote: >> [2] 2005 >> ubuntu@z-nuc-11:~/vgpu-meta/libvirt-stage$ >> *************************************************** >> start libvirt-d >> 2017-03-09 19:04:57.211+0000: 2059: info : libvirt version: 3.1.0 >> 2017-03-09 19:04:57.211+0000: 2059: info : hostname: z-nuc-11.maas >> 2017-03-09 19:04:57.211+0000: 2059: error : qemuMonitorOpenUnix:367 : >> failed to connect to monitor socket: No such process >> 2017-03-09 19:04:57.213+0000: 2059: error : >> virMediatedDeviceGetIOMMUGroupDev:153 : internal error: IOMMU group file /sys/bus/mdev/devices/894f3983-1a36-42b3-b52c-1024aca216be/iommu_group >> is not a symlink > When I saw this error message for the first time in the original thread, I got > confused, since this just checks whether the symlink exists, if it > doesn't, the vfio device probably also doesn't exist (but take this with a > grain of salt, I haven't investigated that deep) and libvirt needs it to pass > it onto qemu command line. I hit this issue once by accident in the past and at > that time I didn't understand what caused it, but after a reboot it was gone. > So seeing it here it caught my eye and I investigated it last week. What I > found out was that it's caused by the vfio-mdev module not being loaded > automatically as a dependency. I solved it by autoloading the module on system > boot. So this is not a libvirt issue, but just for a reference, there is a BZ > on this [1]. it's does not show up every time. might be my bad. > >> 2017-03-09 19:04:57.213+0000: 2003: info : libvirt version: 3.1.0 >> 2017-03-09 19:04:57.213+0000: 2003: info : hostname: z-nuc-11.maas >> 2017-03-09 19:04:57.213+0000: 2003: error : virNetSocketReadWire:1800 : >> End of file while reading data: Input/output error > I suppose this corresponds to the problem above, do you hit this error if you > work around the vfio-mdev module problem described above? > >> the screen call trace while start the VM (same for Ubuntu, Win10 etc) ====================================================== >> >> ubuntu@z-nuc-11:~/vgpu-meta/libvirt-stage$ myvirsh start vgpu-ubuntu >> 2017-03-09 19:06:50.483+0000: 2232: info : libvirt version: 3.1.0 >> 2017-03-09 19:06:50.483+0000: 2232: info : hostname: z-nuc-11.maas >> 2017-03-09 19:06:50.483+0000: 2232: warning : qemuDomainObjTaint:4056 : >> Domain id=1 name='vgpu-ubuntu' uuid=972b5e38-0437-11e7-8f97-d36dba74552d >> is tainted: high-privileges >> 2017-03-09 19:06:50.819+0000: 2204: info : libvirt version: 3.1.0 >> 2017-03-09 19:06:50.819+0000: 2232: warning : virDomainAuditHostdev:456 >> : Unexpected hostdev type while encoding audit message: 4 > This one's interesting, again, are you able to hit the error when you work > around the missing vfio-mdev module? I'll have a look at this if you actually > can hit the error, even if the XML is correct. the module should be there, the domain is running and virtual gpu is working. ubuntu@z-nuc-11:~/vgpu-meta/libvirt-stage$ sudo lsmod| grep mdev vfio_mdev 16384 1 mdev 20480 2 kvmgt,vfio_mdev vfio 28672 6 vfio_iommu_type1,kvmgt,vfio_mdev on the v3, the trace still there. refer to my "test by" email on mdev-next. Regards Yongli He > > I posted v3 of the series and also created a new branch 'mdev-next' on my > github [2]. I dropped the attribute 'type' from the source address element, so > follow the example in the updated docs. > > Thanks for giving it a try. > Erik > > [1] https://bugzilla.redhat.com/show_bug.cgi?id=1420572 > [2] https://github.com/eskultety/libvirt/commits/mdev-next -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
© 2016 - 2024 Red Hat, Inc.