From nobody Sat May 18 04:13:32 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass(p=reject dis=none) header.from=citrix.com ARC-Seal: i=1; a=rsa-sha256; t=1705003933; cv=none; d=zohomail.com; s=zohoarc; b=Ou3RpFH1Rb6sQxMJfup+f+5OBLJTQ4UNd8spJh2PJTdbbvAHLvCKx7LoAPVcP6+1S5cdyF9Fefogp/JiKPoPT2Z3J7ji8QsXPTu/oZ0N7Vyer150Prhln9ITwyXBPQ68Imin6JV5+ylO8iIkKQC5YRizBUViyHDCIZ7bavvuBGU= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1705003933; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=5YevrIboOLEfPxbJii1oBpj+PqGYrUR/U2/YMQT+C2k=; b=JX0F+4ZC3hNXyk+FdW5cX+GplxZZK7WDIFmQ3qE6R20s4q4WpAX8Es0t6vKUia6/mxKCNwOl4sg/1FEIomrUBNQODuE+9wbdxqRT9S/Q0L6zm3dwm9S9qiWJFLAUcaWpwg3rD+UD1f0gs6tL3ryZ6pciSYKUHFgtWk8f4OI9cbA= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass header.from= (p=reject dis=none) Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1705003933120535.1109115761691; Thu, 11 Jan 2024 12:12:13 -0800 (PST) Received: from list by lists.xenproject.org with outflank-mailman.666521.1037213 (Exim 4.92) (envelope-from ) id 1rO1P2-0006Vm-66; Thu, 11 Jan 2024 20:11:48 +0000 Received: by outflank-mailman (output) from mailman id 666521.1037213; Thu, 11 Jan 2024 20:11:48 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1rO1P2-0006Vf-37; Thu, 11 Jan 2024 20:11:48 +0000 Received: by outflank-mailman (input) for mailman id 666521; Thu, 11 Jan 2024 20:11:47 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1rO1P1-0006VZ-4n for xen-devel@lists.xenproject.org; Thu, 11 Jan 2024 20:11:47 +0000 Received: from esa4.hc3370-68.iphmx.com (esa4.hc3370-68.iphmx.com [216.71.155.144]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id a26167c5-b0bd-11ee-9b0f-b553b5be7939; Thu, 11 Jan 2024 21:11:43 +0100 (CET) X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: a26167c5-b0bd-11ee-9b0f-b553b5be7939 DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=citrix.com; s=securemail; t=1705003903; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=vFvoMsqyWFHreX8Y6trU3e3hv4EtiqIn/i79xDFGmac=; b=BRk1p8zv6fjsS3/crXgZJc6fxw2p86jl+MxcvqiuQUmls3JJCDzE+5FJ WuMB01AKs1pWKvl3muSGXntdZFEQ7Te9UacKfk85MeLYeRpGoCK8WdQlO fA/qKV++8madqfRxS/b+64/atC9z8yU5ulmaXNGDDWrNF39dbZqO9rkLg s=; X-CSE-ConnectionGUID: Z6gQJUwJRXSzUt1GtVnlpQ== X-CSE-MsgGUID: XBKvECL9QaqVW/qcQqhyGA== Authentication-Results: esa4.hc3370-68.iphmx.com; dkim=none (message not signed) header.i=none X-SBRS: 4.0 X-MesageID: 132139168 X-Ironport-Server: esa4.hc3370-68.iphmx.com X-Remote-IP: 162.221.159.70 X-Policy: $RELAYED X-ThreatScanner-Verdict: Negative IronPort-Data: A9a23:HbVQR6wdTe9Pyz4M73p6t+dsxirEfRIJ4+MujC+fZmUNrF6WrkUBx jBLDG2Da/zZZWTxeosibY3l9B8Pup/Vx981SgZpqiAxQypGp/SeCIXCJC8cHc8wwu7rFxs7s ppEOrEsCOhuExcwcz/0auCJQUFUjPzOHvykTrecZkidfCc8IA85kxVvhuUltYBhhNm9Emult Mj75sbSIzdJ4RYtWo4vw/zF8E8HUMja4mtC4gRnPKkT5zcyqlFOZH4hDfDpR5fHatE88t6SH 47r0Ly/92XFyBYhYvvNfmHTKxBirhb6ZGBiu1IOM0SQqkEqSh8ai87XAME0e0ZP4whlqvgqo Dl7WT5cfi9yVkHEsLx1vxC1iEiSN4UekFPMCSDXXcB+UyQq2pYjqhljJBheAGEWxgp4KVMQ+ sU6FS8vVCC4ld+qmZ2ndO9onv12eaEHPKtH0p1h5TTQDPJgSpHfWaTao9Rf2V/chOgXQ6yYP ZBAL2MyPVKfO3WjOX9OYH46tM6uimPybHtzr1WNqLBsy2PS0BZwwP7mN9+9ltmiHJ4Pxx3J/ zyWl4j/KiEwF8Ky4h7Uy3u1uNCIhCjCQYIDO5Tto5aGh3XMnzdOWXX6T2CTsfS/z0KzRd9bA 0gV4TY167g/8lSxSdvwVAH+p2SL1jYfXNxKGuF89wCJyYLT+Q+SAmVCRTlEAPQEnsIrQT0h1 neSgsjkQzdotdW9Vna15rqS6zSoNkAowXQqPHFeC1Ffup+6/dB10Uqnostf/LCdvIP6IBPN3 BKwnBMsuqlKotI10aGjxAWS696znaQlXjLZ9y2OATr5t1IhOtL4D7FE/2Q3+hqpEWp4crVil CJd8yRmxLpSZaxhbQTUKAn3IJmn5uyeLBrXikN1Ep8q+lyFoiH7IdkBumEhfRw4bq7onAMFh 2eK5GtsCGJ7ZiP2PcebnartYyjV8UQQPYu8Da2FBja/SpNwaBWG7ElTib24hgjQfLwXufhnY /+zKJ/8ZUv2/Iw7lFJasc9Bi+50rs3/rEuPLa3GI+OPi+PDNC7PFexZbzNjrIkRtcu5nekcy P4HX+Pi9vmVeLSWjvX/mWLLEW03EA== IronPort-HdrOrdr: A9a23:oCb/zqiiwEcBJgrbXYSNiDaBVnBQXuIji2hC6mlwRA09TySZ// rBoB19726MtN9xYgBHpTnuAsm9qB/nmaKdpLNhWItKPzOW31dATrsSjrcKqgeIc0aVm9K1l5 0QF5SWYOeAdWSS5vya3ODXKbkdKaG8gcKVuds= X-Talos-CUID: =?us-ascii?q?9a23=3AarVSNmqixpPXavSW+Oe3n9jmUfJ7fUT/8WjyGX+?= =?us-ascii?q?DG1t1EZKJd1mU4Joxxg=3D=3D?= X-Talos-MUID: 9a23:ZUiXgQv4xbmWCaDUlM2ntBtBKupDx/WXAVkzrtIfsPCdNT1fEmLI X-IronPort-AV: E=Sophos;i="6.04,187,1695700800"; d="scan'208";a="132139168" From: Andrew Cooper To: Xen-devel CC: Andrew Cooper , Konrad Rzeszutek Wilk , Ross Lagerwall , "Jan Beulich" , =?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= , Wei Liu Subject: [PATCH v1-alt] xen/livepatch: Make check_for_livepatch_work() faster in the common case Date: Thu, 11 Jan 2024 20:11:29 +0000 Message-ID: <20240111201129.4010175-1-andrew.cooper3@citrix.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20231222220045.2840714-1-andrew.cooper3@citrix.com> References: <20231222220045.2840714-1-andrew.cooper3@citrix.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @citrix.com) X-ZM-MESSAGEID: 1705003933866100001 When livepatching is enabled, this function is used all the time. Really do check the fastpath first, and annotate it likely() as this is the right ans= wer 100% of the time (to many significant figures). This cuts out 3 pointer dereferences in the "nothing to do path". However, GCC still needs some help to persuade it not to set the full stack frame (6 spilled registers, 3 slots of locals) even on the fastpath. Create a new check_for_livepatch_work() with the fastpath only, and make the "new" do_livepatch_work() noinline. This causes the fastpath to need no st= ack frame, making it faster still. Signed-off-by: Andrew Cooper Reviewed-by: Jan Beulich --- CC: Konrad Rzeszutek Wilk CC: Ross Lagerwall CC: Jan Beulich CC: Roger Pau Monn=C3=A9 CC: Wei Liu v1-alt: * Manually split the functions. Experimenting with __attribute__((cold)) was disappointing. Vs this patch,= it creates an extra check_for_livepatch_work.cold function(and section) which = is just `jmp do_livepatch_work`. --- xen/common/livepatch.c | 17 ++++++++++++----- 1 file changed, 12 insertions(+), 5 deletions(-) diff --git a/xen/common/livepatch.c b/xen/common/livepatch.c index 1209fea2566c..2c4b84382798 100644 --- a/xen/common/livepatch.c +++ b/xen/common/livepatch.c @@ -1693,7 +1693,7 @@ static int livepatch_spin(atomic_t *counter, s_time_t= timeout, * The main function which manages the work of quiescing the system and * patching code. */ -void check_for_livepatch_work(void) +static void noinline do_livepatch_work(void) { #define ACTION(x) [LIVEPATCH_ACTION_##x] =3D #x static const char *const names[] =3D { @@ -1711,10 +1711,6 @@ void check_for_livepatch_work(void) !is_idle_domain(current->sched_unit->domain) ) return; =20 - /* Fast path: no work to do. */ - if ( !per_cpu(work_to_do, cpu ) ) - return; - smp_rmb(); /* In case we aborted, other CPUs can skip right away. */ if ( !livepatch_work.do_work ) @@ -1864,6 +1860,17 @@ void check_for_livepatch_work(void) } } =20 +void check_for_livepatch_work(void) +{ + unsigned int cpu =3D smp_processor_id(); + + /* Fast path: no work to do. */ + if ( likely(!per_cpu(work_to_do, cpu)) ) + return; + + do_livepatch_work(); +} + /* * Only allow dependent payload is applied on top of the correct * build-id. --=20 2.30.2