From nobody Fri Nov 29 16:39:12 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass(p=quarantine dis=none) header.from=suse.com ARC-Seal: i=1; a=rsa-sha256; t=1632739715; cv=none; d=zohomail.com; s=zohoarc; b=ejAFDey/6CE1sd5gQR4M0EBe+P978QRO5GirAPRc2SpLfr2b+n1Sl4E0ZRBSYI59OFLjGuu7N+/SAazHq/k7NTdPhQvr00A4pWTmPrGRy0sl+5So92xFGGWMEUl+9W4eJoZLkGJZFVRi2TBZA1cEywbFC1b94d8cKTM7RNdKrH4= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1632739715; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=8hffVnuWEiEzQ5SGP29hjShMeVnOse/BmKMl/NYs2rY=; b=PQvaddlHcWWJt01mbnRWcPo3LtbLYQLjgH0xXG0sNRdoXOLxqti3bi0jSPLjNDB+ZiwaXlMnl2zh/ynVP2n465IqXcxBaBeSTyU9uDYOh1Vb/aad2YT5/Gewhv8PL+jhmGE25sqARhZhnvnum3cxQ7r8rVSXGmWzIbgUFoPtz1g= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1632739715690762.2693511597738; Mon, 27 Sep 2021 03:48:35 -0700 (PDT) Received: from list by lists.xenproject.org with outflank-mailman.196702.349668 (Exim 4.92) (envelope-from ) id 1mUoBL-0003ud-I5; Mon, 27 Sep 2021 10:48:23 +0000 Received: by outflank-mailman (output) from mailman id 196702.349668; Mon, 27 Sep 2021 10:48:23 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1mUoBL-0003uU-DP; Mon, 27 Sep 2021 10:48:23 +0000 Received: by outflank-mailman (input) for mailman id 196702; Mon, 27 Sep 2021 10:48:22 +0000 Received: from us1-rack-iad1.inumbo.com ([172.99.69.81]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1mUoBK-0003Gz-7n for xen-devel@lists.xenproject.org; Mon, 27 Sep 2021 10:48:22 +0000 Received: from smtp-out2.suse.de (unknown [195.135.220.29]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS id ea33450a-7757-403b-bd59-0628b7106e2d; Mon, 27 Sep 2021 10:48:16 +0000 (UTC) Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id E1D0C200DD; Mon, 27 Sep 2021 10:48:15 +0000 (UTC) Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id BEB8C13A7F; Mon, 27 Sep 2021 10:48:15 +0000 (UTC) Received: from dovecot-director2.suse.de ([192.168.254.65]) by imap2.suse-dmz.suse.de with ESMTPSA id EJhrLW+hUWHBKgAAMHmgww (envelope-from ); Mon, 27 Sep 2021 10:48:15 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: ea33450a-7757-403b-bd59-0628b7106e2d DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1; t=1632739695; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=8hffVnuWEiEzQ5SGP29hjShMeVnOse/BmKMl/NYs2rY=; b=j3d/BkoySyIIS8qrc3ZLM5AapZ7EJIhtiAGn/Ox/lWGCYahv7vWz7Fru/WdRsFXYywTGJw CJCEEOeBbKH0MJ16k3TtBQwFgBWhYQyj4QHuJHteXUl93N17YA1GI1RWh5ekszUdw6Oc+7 Nv/f7Q5iejuENOatfN7yRkURIKvUc9o= From: Juergen Gross To: xen-devel@lists.xenproject.org Cc: Juergen Gross , Ian Jackson , Wei Liu Subject: [PATCH v4 1/2] tools/xenstore: set oom score for xenstore daemon on Linux Date: Mon, 27 Sep 2021 12:48:12 +0200 Message-Id: <20210927104813.19772-2-jgross@suse.com> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20210927104813.19772-1-jgross@suse.com> References: <20210927104813.19772-1-jgross@suse.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @suse.com) X-ZM-MESSAGEID: 1632739716652100007 Content-Type: text/plain; charset="utf-8" Xenstored is absolutely mandatory for a Xen host and it can't be restarted, so being killed by OOM-killer in case of memory shortage is to be avoided. Set /proc/$pid/oom_score_adj (if available) per default to -500 (this translates to 50% of dom0 memory size) in order to allow xenstored to use large amounts of memory without being killed. The percentage of dom0 memory above which the oom killer is allowed to kill xenstored can be set via XENSTORED_OOM_MEM_THRESHOLD in xencommons. Make sure the pid file isn't a left-over from a previous run delete it before starting xenstored. Signed-off-by: Juergen Gross Reviewed-by: Ian Jackson --- V2: - set oom score from launch script (Julien Grall) - split off open file descriptor limit setting (Julien Grall) V3: - make oom killer threshold configurable (Julien Grall) V4: - extend comment (Ian Jackson) --- tools/hotplug/Linux/init.d/sysconfig.xencommons.in | 9 +++++++++ tools/hotplug/Linux/launch-xenstore.in | 6 ++++++ 2 files changed, 15 insertions(+) diff --git a/tools/hotplug/Linux/init.d/sysconfig.xencommons.in b/tools/hot= plug/Linux/init.d/sysconfig.xencommons.in index 00cf7f91d4..b83101ab7e 100644 --- a/tools/hotplug/Linux/init.d/sysconfig.xencommons.in +++ b/tools/hotplug/Linux/init.d/sysconfig.xencommons.in @@ -48,6 +48,15 @@ XENSTORED_ARGS=3D # Only evaluated if XENSTORETYPE is "daemon". #XENSTORED_TRACE=3D[yes|on|1] =20 +## Type: integer +## Default: 50 +# +# Percentage of dom0 memory size the xenstore daemon can use before the +# OOM killer is allowed to kill it. +# The specified value is multiplied by -10 and echoed to +# /proc/PID/oom_score_adj. +#XENSTORED_OOM_MEM_THRESHOLD=3D50 + ## Type: string ## Default: @LIBEXEC@/boot/xenstore-stubdom.gz # diff --git a/tools/hotplug/Linux/launch-xenstore.in b/tools/hotplug/Linux/l= aunch-xenstore.in index 019f9d6f4d..1747c96065 100644 --- a/tools/hotplug/Linux/launch-xenstore.in +++ b/tools/hotplug/Linux/launch-xenstore.in @@ -59,11 +59,17 @@ test -f @CONFIG_DIR@/@CONFIG_LEAF_DIR@/xencommons && . = @CONFIG_DIR@/@CONFIG_LEAF echo "No xenstored found" exit 1 } + [ -z "$XENSTORED_OOM_MEM_THRESHOLD" ] || XENSTORED_OOM_MEM_THRESHOLD=3D50 + XS_OOM_SCORE=3D-$(($XENSTORED_OOM_MEM_THRESHOLD * 10)) + + rm -f @XEN_RUN_DIR@/xenstored.pid =20 echo -n Starting $XENSTORED... $XENSTORED --pid-file @XEN_RUN_DIR@/xenstored.pid $XENSTORED_ARGS =20 systemd-notify --booted 2>/dev/null || timeout_xenstore $XENSTORED || exi= t 1 + XS_PID=3D`cat @XEN_RUN_DIR@/xenstored.pid` + echo $XS_OOM_SCORE >/proc/$XS_PID/oom_score_adj =20 exit 0 } --=20 2.26.2 From nobody Fri Nov 29 16:39:12 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass(p=quarantine dis=none) header.from=suse.com ARC-Seal: i=1; a=rsa-sha256; t=1632739714; cv=none; d=zohomail.com; s=zohoarc; b=T/NM6P+fE3r1sk3FhS7mfrfUxU0zpRPT2tQHcGGrNh/zDdgjvjVWmN7nn0j4Y/YFVakd8m5yqSFFCJeUNrit6M8eaytE8a72D4GqpfrQkmsz8Ro0eVKrWEilQE1hEkwjiA2Op7MIa9oNE6bCoJ+i1cHgvVexjzqUEWCbBPJ/FOo= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1632739714; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=0bTrNbQ0miBKyNabZ7OBv568TXSZHmfPJwYLlLSwrss=; b=F0bNCxkvnBfTenh/1o5AFhI4tS7PrcIDgYcyLb9LhTqG5KkKgNzIAfFySi5jThJNw0tWzo+j1nUBAMoUzRYgKYaLZcmE4R4NS7IGxqB4qmhtau2xCaLzeakKu7LCD/9hD598sIVlJx/0iyTwd68HCysGof/0gMx6/CXgdel39fc= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 16327397140511.1779350673203908; Mon, 27 Sep 2021 03:48:34 -0700 (PDT) Received: from list by lists.xenproject.org with outflank-mailman.196701.349652 (Exim 4.92) (envelope-from ) id 1mUoBH-0003Ks-Ap; Mon, 27 Sep 2021 10:48:19 +0000 Received: by outflank-mailman (output) from mailman id 196701.349652; Mon, 27 Sep 2021 10:48:19 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1mUoBH-0003Jh-2A; Mon, 27 Sep 2021 10:48:19 +0000 Received: by outflank-mailman (input) for mailman id 196701; Mon, 27 Sep 2021 10:48:17 +0000 Received: from all-amaz-eas1.inumbo.com ([34.197.232.57] helo=us1-amaz-eas2.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1mUoBF-0003HF-Nf for xen-devel@lists.xenproject.org; Mon, 27 Sep 2021 10:48:17 +0000 Received: from smtp-out1.suse.de (unknown [195.135.220.28]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS id 6b8e251c-1f80-11ec-bc44-12813bfff9fa; Mon, 27 Sep 2021 10:48:16 +0000 (UTC) Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id 148E22209B; Mon, 27 Sep 2021 10:48:16 +0000 (UTC) Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id E5D0013F4B; Mon, 27 Sep 2021 10:48:15 +0000 (UTC) Received: from dovecot-director2.suse.de ([192.168.254.65]) by imap2.suse-dmz.suse.de with ESMTPSA id MBL/Nm+hUWHBKgAAMHmgww (envelope-from ); Mon, 27 Sep 2021 10:48:15 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 6b8e251c-1f80-11ec-bc44-12813bfff9fa DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1; t=1632739696; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=0bTrNbQ0miBKyNabZ7OBv568TXSZHmfPJwYLlLSwrss=; b=Pl/LD+1gSxuMVta+tt/eO4ZEzBzgRAh0YlzD8rfBG3ybn9WUnHRuV3h1n4vB6BUPDAaSZk CGlvtWQxwq+0Ata56n89R9ybSNeVv45CwdRIN0EcYkL0piIPBNEHV363QHagqSrrkaH3YR 4GJa+Qx4t7+iXWBRHSP5Jrfls7TgDjw= From: Juergen Gross To: xen-devel@lists.xenproject.org Cc: Juergen Gross , Ian Jackson , Wei Liu Subject: [PATCH v4 2/2] tools/xenstore: set open file descriptor limit for xenstored Date: Mon, 27 Sep 2021 12:48:13 +0200 Message-Id: <20210927104813.19772-3-jgross@suse.com> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20210927104813.19772-1-jgross@suse.com> References: <20210927104813.19772-1-jgross@suse.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @suse.com) X-ZM-MESSAGEID: 1632739714896100002 Content-Type: text/plain; charset="utf-8" Add a configuration item for the maximum number of open file descriptors xenstored should be allowed to have. The default should be "unlimited" in order not to restrict xenstored in the number of domains it can support, but unfortunately the prlimit command requires specification of a real value for the number of files, so use 262144 as the default value. As an aid for the admin configuring the value add a comment specifying the common needs of xenstored for the different domain types. Signed-off-by: Juergen Gross --- V2: - set ulimit form launch script (Julien Grall) - split off from original patch (Julien Grall) V4: - switch to directly configuring the limit of file descriptors instead of domains (Ian Jackson) --- tools/hotplug/Linux/init.d/sysconfig.xencommons.in | 13 +++++++++++++ tools/hotplug/Linux/launch-xenstore.in | 2 ++ 2 files changed, 15 insertions(+) diff --git a/tools/hotplug/Linux/init.d/sysconfig.xencommons.in b/tools/hot= plug/Linux/init.d/sysconfig.xencommons.in index b83101ab7e..ad020b7a50 100644 --- a/tools/hotplug/Linux/init.d/sysconfig.xencommons.in +++ b/tools/hotplug/Linux/init.d/sysconfig.xencommons.in @@ -32,6 +32,19 @@ # Changing this requires a reboot to take effect. #XENSTORED=3D@XENSTORED@ =20 +## Type: integer +## Default: 262144 +# +# Select maximum number of file descriptors xenstored is allowed to have +# opened at one time. +# For each HVM domain xenstored might need up to 5 open file descriptors, +# PVH and PV domains will require up to 3 open file descriptors. Additiona= lly +# 20-30 file descriptors will be opened for internal uses. The default of +# 262144 allows for about 8 open files for the theoretical maximum of 32752 +# domains. +# Only evaluated if XENSTORETYPE is "daemon". +#XENSTORED_MAX_OPEN_FDS=3D262144 + ## Type: string ## Default: "" # diff --git a/tools/hotplug/Linux/launch-xenstore.in b/tools/hotplug/Linux/l= aunch-xenstore.in index 1747c96065..2bc41bb4f0 100644 --- a/tools/hotplug/Linux/launch-xenstore.in +++ b/tools/hotplug/Linux/launch-xenstore.in @@ -54,6 +54,7 @@ test -f @CONFIG_DIR@/@CONFIG_LEAF_DIR@/xencommons && . @C= ONFIG_DIR@/@CONFIG_LEAF =20 [ "$XENSTORETYPE" =3D "daemon" ] && { [ -z "$XENSTORED_TRACE" ] || XENSTORED_ARGS=3D"$XENSTORED_ARGS -T @XEN_LO= G_DIR@/xenstored-trace.log" + [ -z "$XENSTORED_MAX_OPEN_FDS" ] && XENSTORED_MAX_OPEN_FDS=3D262144 [ -z "$XENSTORED" ] && XENSTORED=3D@XENSTORED@ [ -x "$XENSTORED" ] || { echo "No xenstored found" @@ -70,6 +71,7 @@ test -f @CONFIG_DIR@/@CONFIG_LEAF_DIR@/xencommons && . @C= ONFIG_DIR@/@CONFIG_LEAF systemd-notify --booted 2>/dev/null || timeout_xenstore $XENSTORED || exi= t 1 XS_PID=3D`cat @XEN_RUN_DIR@/xenstored.pid` echo $XS_OOM_SCORE >/proc/$XS_PID/oom_score_adj + prlimit --pid $XS_PID --nofile=3D$XENSTORED_MAX_OPEN_FDS =20 exit 0 } --=20 2.26.2