From nobody Fri Apr 19 12:30:52 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=none (zohomail.com: 192.237.175.120 is neither permitted nor denied by domain of lists.xenproject.org) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=fail(p=none dis=none) header.from=citrix.com ARC-Seal: i=1; a=rsa-sha256; t=1579034222; cv=none; d=zohomail.com; s=zohoarc; b=Std7ihL1R4cLZwl9tp6S1Y8t7Wg0vTFYlavObHrmEEtCe0A8kimHnDOxVZ1kXbJreNytGFkWplgjEyCLcT7AiIjVP+rPtb8lhzIrq3KyXNATaEfsSVN0zpY3PaWfhWu4eynhQk1oBnB3eS2KzLO+9AukvqkUmVN1LMJb3Hfjq0U= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1579034222; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:Sender:Subject:To; bh=Wa2yKrMeytksfd1/aCIxFxJHq6Ycp7C7r1M59CSG4Dc=; b=geQTTFDVidGg8b9z39saxh+2yeARZwt8rmZeMv94RWh30+Cl7tsp/V2BpCtqR4XXQi4NZ3DMq78IQ+L9jOqny/2GcjE5i5rv94JrIfUqPq1oOyF4CwkPKnV7peUIhcSnyqgpqKIgECwCiEEsd+xfsYmrBUTq8Spnw/VqTaA0kJ4= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=fail; spf=none (zohomail.com: 192.237.175.120 is neither permitted nor denied by domain of lists.xenproject.org) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1579034222915523.8123308952794; Tue, 14 Jan 2020 12:37:02 -0800 (PST) Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1irSus-0000py-LQ; Tue, 14 Jan 2020 20:35:58 +0000 Received: from all-amaz-eas1.inumbo.com ([34.197.232.57] helo=us1-amaz-eas2.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1irSuq-0000pt-EW for xen-devel@lists.xenproject.org; Tue, 14 Jan 2020 20:35:56 +0000 Received: from esa6.hc3370-68.iphmx.com (unknown [216.71.155.175]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS id 73af18c6-370d-11ea-8467-12813bfff9fa; Tue, 14 Jan 2020 20:35:50 +0000 (UTC) X-Inumbo-ID: 73af18c6-370d-11ea-8467-12813bfff9fa DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=citrix.com; s=securemail; t=1579034150; h=from:to:cc:subject:date:message-id:mime-version: content-transfer-encoding; bh=9wiXdmQbd1rbUfY9H44kInJbaR4A3j4phCEOp47yt1o=; b=e4GQ6kj3YTn71ro8fDD9qCHs23zKEjp+0Boh4AK6yW+ed9lW4VF1mSoz pej8Ja1bfqfrqfmPk4e9frs4rlNGZ4ybyBzQBsP+v38Uf8mEw+ip2plkg 43XQK/PWsHVknZf8dUuDOkzkkn7+lplsq2E7I+VWTUAKq8oNIKDB1fjHb w=; Authentication-Results: esa6.hc3370-68.iphmx.com; dkim=none (message not signed) header.i=none; spf=None smtp.pra=andrew.cooper3@citrix.com; spf=Pass smtp.mailfrom=Andrew.Cooper3@citrix.com; spf=None smtp.helo=postmaster@mail.citrix.com Received-SPF: none (zohomail.com: 192.237.175.120 is neither permitted nor denied by domain of lists.xenproject.org) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Received-SPF: None (esa6.hc3370-68.iphmx.com: no sender authenticity information available from domain of andrew.cooper3@citrix.com) identity=pra; client-ip=162.221.158.21; receiver=esa6.hc3370-68.iphmx.com; envelope-from="Andrew.Cooper3@citrix.com"; x-sender="andrew.cooper3@citrix.com"; x-conformance=sidf_compatible Received-SPF: Pass (esa6.hc3370-68.iphmx.com: domain of Andrew.Cooper3@citrix.com designates 162.221.158.21 as permitted sender) identity=mailfrom; client-ip=162.221.158.21; receiver=esa6.hc3370-68.iphmx.com; envelope-from="Andrew.Cooper3@citrix.com"; x-sender="Andrew.Cooper3@citrix.com"; x-conformance=sidf_compatible; x-record-type="v=spf1"; x-record-text="v=spf1 ip4:209.167.231.154 ip4:178.63.86.133 ip4:195.66.111.40/30 ip4:85.115.9.32/28 ip4:199.102.83.4 ip4:192.28.146.160 ip4:192.28.146.107 ip4:216.52.6.88 ip4:216.52.6.188 ip4:162.221.158.21 ip4:162.221.156.83 ip4:168.245.78.127 ~all" Received-SPF: None (esa6.hc3370-68.iphmx.com: no sender authenticity information available from domain of postmaster@mail.citrix.com) identity=helo; client-ip=162.221.158.21; receiver=esa6.hc3370-68.iphmx.com; envelope-from="Andrew.Cooper3@citrix.com"; x-sender="postmaster@mail.citrix.com"; x-conformance=sidf_compatible IronPort-SDR: iymLlbW6Wl74yAb08Zj2pqNR98o1Yb7kY163aMvUiVHCqYpZzXThPSKiGIP9rF4MALZ6ztjosr G43leMgpz/mVrxCzSppzmX/pZJEpbxaX4Mr5Xb9z/Xyx1APWe923Bwea05RmBN+nPzDbMBSQyK RRWTbRCCFb8kgrRlC6Qo/vL+kcoNJ57ak6gJ11Bwx28dhQ+8iPhncbmXu4uGgrdEDXkJ9Diljy JEh3ojvIbuZHQCdQ/kZbTaJZO+6eRt1oc2Dcht19HgTtUVxyETZY5GKTIgDwt70x3Ui7//O0KV BFo= X-SBRS: 2.7 X-MesageID: 11343145 X-Ironport-Server: esa6.hc3370-68.iphmx.com X-Remote-IP: 162.221.158.21 X-Policy: $RELAYED X-IronPort-AV: E=Sophos;i="5.70,320,1574139600"; d="scan'208";a="11343145" From: Andrew Cooper To: Xen-devel Date: Tue, 14 Jan 2020 20:35:45 +0000 Message-ID: <20200114203545.8897-1-andrew.cooper3@citrix.com> X-Mailer: git-send-email 2.11.0 MIME-Version: 1.0 Subject: [Xen-devel] [PATCH] xen/list: Remove prefetching X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Stefano Stabellini , Julien Grall , Wei Liu , Andrew Cooper , Jan Beulich , Volodymyr Babchuk , =?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) Xen inherited its list infrastructure from Linux. One area where has fallen behind is that of prefetching, which as it turns out is a performance penal= ty in most cases. Prefetch of NULL on x86 is now widely measured to have glacial performance properties, and will unconditionally hit on every hlist use due to the termination condition. Cross-port the following Linux patches: 75d65a425c (2011) "hlist: remove software prefetching in hlist iterators" e66eed651f (2011) "list: remove prefetching from regular list iterators" c0d15cc7ee (2013) "linked-list: Remove __list_for_each" to Xen, which results in the following net diffstat on x86: add/remove: 0/1 grow/shrink: 27/83 up/down: 576/-1648 (-1072) (The code additions comes from a few now-inlined functions, and slightly different basic block padding.) No functional change. Signed-off-by: Andrew Cooper Acked-by: Julien Grall Reviewed-by: Roger Pau Monn=C3=A9 --- CC: Jan Beulich CC: Wei Liu CC: Roger Pau Monn=C3=A9 CC: Stefano Stabellini CC: Julien Grall CC: Volodymyr Babchuk --- xen/include/xen/list.h | 46 +++++++++++++--------------------------------- 1 file changed, 13 insertions(+), 33 deletions(-) diff --git a/xen/include/xen/list.h b/xen/include/xen/list.h index 1387abb211..dc5a8c461b 100644 --- a/xen/include/xen/list.h +++ b/xen/include/xen/list.h @@ -42,9 +42,6 @@ struct list_head { #define LIST_HEAD_READ_MOSTLY(name) \ struct list_head __read_mostly name =3D LIST_HEAD_INIT(name) =20 -/* Do not move this ahead of the struct list_head definition! */ -#include - static inline void INIT_LIST_HEAD(struct list_head *list) { list->next =3D list; @@ -455,20 +452,6 @@ static inline void list_splice_init(struct list_head *= list, * @head: the head for your list. */ #define list_for_each(pos, head) \ - for (pos =3D (head)->next; prefetch(pos->next), pos !=3D (head); = \ - pos =3D pos->next) - -/** - * __list_for_each - iterate over a list - * @pos: the &struct list_head to use as a loop cursor. - * @head: the head for your list. - * - * This variant differs from list_for_each() in that it's the - * simplest possible list iteration code, no prefetching is done. - * Use this for code that knows the list to be very short (empty - * or 1 entry) most of the time. - */ -#define __list_for_each(pos, head) \ for (pos =3D (head)->next; pos !=3D (head); pos =3D pos->next) =20 /** @@ -477,8 +460,7 @@ static inline void list_splice_init(struct list_head *l= ist, * @head: the head for your list. */ #define list_for_each_prev(pos, head) \ - for (pos =3D (head)->prev; prefetch(pos->prev), pos !=3D (head); = \ - pos =3D pos->prev) + for (pos =3D (head)->prev; pos !=3D (head); pos =3D pos->prev) =20 /** * list_for_each_safe - iterate over a list safe against removal of list e= ntry @@ -509,7 +491,7 @@ static inline void list_splice_init(struct list_head *l= ist, */ #define list_for_each_entry(pos, head, member) \ for (pos =3D list_entry((head)->next, typeof(*pos), member); \ - prefetch(pos->member.next), &pos->member !=3D (head); \ + &pos->member !=3D (head); \ pos =3D list_entry(pos->member.next, typeof(*pos), member)) =20 /** @@ -520,7 +502,7 @@ static inline void list_splice_init(struct list_head *l= ist, */ #define list_for_each_entry_reverse(pos, head, member) \ for (pos =3D list_entry((head)->prev, typeof(*pos), member); \ - prefetch(pos->member.prev), &pos->member !=3D (head); \ + &pos->member !=3D (head); \ pos =3D list_entry(pos->member.prev, typeof(*pos), member)) =20 /** @@ -547,7 +529,7 @@ static inline void list_splice_init(struct list_head *l= ist, */ #define list_for_each_entry_continue(pos, head, member) \ for (pos =3D list_entry(pos->member.next, typeof(*pos), member); \ - prefetch(pos->member.next), &pos->member !=3D (head); \ + &pos->member !=3D (head); \ pos =3D list_entry(pos->member.next, typeof(*pos), member)) =20 /** @@ -560,7 +542,7 @@ static inline void list_splice_init(struct list_head *l= ist, * Iterate over list of given type, continuing from current position. */ #define list_for_each_entry_from(pos, head, member) \ - for (; prefetch(pos->member.next), &pos->member !=3D (head); \ + for (; &pos->member !=3D (head); \ pos =3D list_entry(pos->member.next, typeof(*pos), member)) =20 /** @@ -635,7 +617,7 @@ static inline void list_splice_init(struct list_head *l= ist, */ #define list_for_each_rcu(pos, head) \ for (pos =3D (head)->next; \ - prefetch(rcu_dereference(pos)->next), pos !=3D (head); \ + rcu_dereference(pos) !=3D (head); \ pos =3D pos->next) =20 #define __list_for_each_rcu(pos, head) \ @@ -672,8 +654,7 @@ static inline void list_splice_init(struct list_head *l= ist, */ #define list_for_each_entry_rcu(pos, head, member) \ for (pos =3D list_entry((head)->next, typeof(*pos), member); \ - prefetch(rcu_dereference(pos)->member.next), \ - &pos->member !=3D (head); \ + &rcu_dereference(pos)->member !=3D (head); \ pos =3D list_entry(pos->member.next, typeof(*pos), member)) =20 /** @@ -689,7 +670,7 @@ static inline void list_splice_init(struct list_head *l= ist, */ #define list_for_each_continue_rcu(pos, head) \ for ((pos) =3D (pos)->next; \ - prefetch(rcu_dereference((pos))->next), (pos) !=3D (head); \ + rcu_dereference(pos) !=3D (head); \ (pos) =3D (pos)->next) =20 /* @@ -918,8 +899,7 @@ static inline void hlist_add_after_rcu(struct hlist_nod= e *prev, #define hlist_entry(ptr, type, member) container_of(ptr,type,member) =20 #define hlist_for_each(pos, head) \ - for (pos =3D (head)->first; pos && ({ prefetch(pos->next); 1; }); \ - pos =3D pos->next) + for (pos =3D (head)->first; pos; pos =3D pos->next) =20 #define hlist_for_each_safe(pos, n, head) \ for (pos =3D (head)->first; pos && ({ n =3D pos->next; 1; }); \ @@ -934,7 +914,7 @@ static inline void hlist_add_after_rcu(struct hlist_nod= e *prev, */ #define hlist_for_each_entry(tpos, pos, head, member) \ for (pos =3D (head)->first; \ - pos && ({ prefetch(pos->next); 1;}) && \ + pos && \ ({ tpos =3D hlist_entry(pos, typeof(*tpos), member); 1;}); \ pos =3D pos->next) =20 @@ -947,7 +927,7 @@ static inline void hlist_add_after_rcu(struct hlist_nod= e *prev, */ #define hlist_for_each_entry_continue(tpos, pos, member) \ for (pos =3D (pos)->next; \ - pos && ({ prefetch(pos->next); 1;}) && \ + pos && \ ({ tpos =3D hlist_entry(pos, typeof(*tpos), member); 1;}); \ pos =3D pos->next) =20 @@ -959,7 +939,7 @@ static inline void hlist_add_after_rcu(struct hlist_nod= e *prev, * @member: the name of the hlist_node within the struct. */ #define hlist_for_each_entry_from(tpos, pos, member) \ - for (; pos && ({ prefetch(pos->next); 1;}) && \ + for (; pos && \ ({ tpos =3D hlist_entry(pos, typeof(*tpos), member); 1;}); \ pos =3D pos->next) =20 @@ -992,7 +972,7 @@ static inline void hlist_add_after_rcu(struct hlist_nod= e *prev, */ #define hlist_for_each_entry_rcu(tpos, pos, head, member) \ for (pos =3D (head)->first; \ - rcu_dereference(pos) && ({ prefetch(pos->next); 1;}) && \ + rcu_dereference(pos) && \ ({ tpos =3D hlist_entry(pos, typeof(*tpos), member); 1;}); \ pos =3D pos->next) =20 --=20 2.11.0 _______________________________________________ Xen-devel mailing list Xen-devel@lists.xenproject.org https://lists.xenproject.org/mailman/listinfo/xen-devel