libvirtd can spawn threads/tasks when creating new domains for
some hypervisors such as Xen's libxl driver, quickly reaching
the cgroups pids controller default TasksMax setting of 512. When
the limit is reached, attempting to create additional domains
results in an error from the cgroups pids controller, e.g.

kernel: [71282.213347] cgroup: fork rejected by pids controller in
/system.slice/libvirtd.service

Depending on domain type and configuration, anywhere from 4-7
threads/tasks may be created by libxl when starting a domain.
In order to support 4096 domains, similar to commit 27cd763500,
increase the TasksMax setting in libvirtd.service to
4096 * 8 = 32768 tasks.

Signed-off-by: Jim Fehlig <jfehlig@suse.com>
---
 daemon/libvirtd.service.in | 5 +++++
 1 file changed, 5 insertions(+)

diff --git a/daemon/libvirtd.service.in b/daemon/libvirtd.service.in
index 899abdf37..fbaf02f3b 100644
--- a/daemon/libvirtd.service.in
+++ b/daemon/libvirtd.service.in
@@ -29,6 +29,11 @@ Restart=on-failure
 # If changing this, also consider virtlogd.service & virtlockd.service
 # limits which are also related to number of guests
 LimitNOFILE=8192
+# The cgroups pids controller can limit the number of tasks started by
+# the daemon, which can limit the number of domains for some hypervisors.
+# A conservative default of 8 tasks per guest results in a TasksMax of
+# 32k to support 4096 guests.
+TasksMax=32768
 
 [Install]
 WantedBy=multi-user.target
-- 
2.11.0

--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list

On Wed, Apr 19, 2017 at 01:29:01PM -0600, Jim Fehlig wrote:
> libvirtd can spawn threads/tasks when creating new domains for
> some hypervisors such as Xen's libxl driver, quickly reaching
> the cgroups pids controller default TasksMax setting of 512. When
> the limit is reached, attempting to create additional domains
> results in an error from the cgroups pids controller, e.g.
> 
> kernel: [71282.213347] cgroup: fork rejected by pids controller in
> /system.slice/libvirtd.service
> 
> Depending on domain type and configuration, anywhere from 4-7
> threads/tasks may be created by libxl when starting a domain.
> In order to support 4096 domains, similar to commit 27cd763500,
> increase the TasksMax setting in libvirtd.service to
> 4096 * 8 = 32768 tasks.
> 
> Signed-off-by: Jim Fehlig <jfehlig@suse.com>
> ---
>  daemon/libvirtd.service.in | 5 +++++
>  1 file changed, 5 insertions(+)
> 
> diff --git a/daemon/libvirtd.service.in b/daemon/libvirtd.service.in
> index 899abdf37..fbaf02f3b 100644
> --- a/daemon/libvirtd.service.in
> +++ b/daemon/libvirtd.service.in
> @@ -29,6 +29,11 @@ Restart=on-failure
>  # If changing this, also consider virtlogd.service & virtlockd.service
>  # limits which are also related to number of guests
>  LimitNOFILE=8192
> +# The cgroups pids controller can limit the number of tasks started by
> +# the daemon, which can limit the number of domains for some hypervisors.
> +# A conservative default of 8 tasks per guest results in a TasksMax of
> +# 32k to support 4096 guests.
> +TasksMax=32768
>  
>  [Install]
>  WantedBy=multi-user.target

ACK


Regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|

--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list