From nobody Sat May 4 08:14:50 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1499925300985300.4126680383109; Wed, 12 Jul 2017 22:55:00 -0700 (PDT) Received: from localhost ([::1]:57300 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dVX5V-0000VT-Lp for importer@patchew.org; Thu, 13 Jul 2017 01:54:57 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:36106) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dVX3v-0007O4-Ew for qemu-devel@nongnu.org; Thu, 13 Jul 2017 01:53:20 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1dVX3u-0006Ig-Kj for qemu-devel@nongnu.org; Thu, 13 Jul 2017 01:53:19 -0400 Received: from [59.151.112.132] (port=25092 helo=heian.cn.fujitsu.com) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dVX3u-0006C3-8Q for qemu-devel@nongnu.org; Thu, 13 Jul 2017 01:53:18 -0400 Received: from unknown (HELO cn.fujitsu.com) ([10.167.33.5]) by heian.cn.fujitsu.com with ESMTP; 13 Jul 2017 13:53:04 +0800 Received: from G08CNEXCHPEKD02.g08.fujitsu.local (unknown [10.167.33.83]) by cn.fujitsu.com (Postfix) with ESMTP id A1CB346B5242; Thu, 13 Jul 2017 13:53:01 +0800 (CST) Received: from localhost.localdomain (10.167.226.56) by G08CNEXCHPEKD02.g08.fujitsu.local (10.167.33.89) with Microsoft SMTP Server (TLS) id 14.3.319.2; Thu, 13 Jul 2017 13:53:00 +0800 X-IronPort-AV: E=Sophos;i="5.22,518,1449504000"; d="scan'208";a="21231271" From: Zhang Chen To: qemu devel , Jason Wang Date: Thu, 13 Jul 2017 13:52:52 +0800 Message-ID: <1499925175-21218-2-git-send-email-zhangchen.fnst@cn.fujitsu.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1499925175-21218-1-git-send-email-zhangchen.fnst@cn.fujitsu.com> References: <1499925175-21218-1-git-send-email-zhangchen.fnst@cn.fujitsu.com> MIME-Version: 1.0 X-Originating-IP: [10.167.226.56] X-yoursite-MailScanner-ID: A1CB346B5242.ACCB7 X-yoursite-MailScanner: Found to be clean X-yoursite-MailScanner-From: zhangchen.fnst@cn.fujitsu.com X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 59.151.112.132 Subject: [Qemu-devel] [PATCH V2 1/4] net/colo-compare.c: Add checkpoint min period to optimize performance X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Li Zhijian , zhanghailiang , Zhang Chen Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" If colo-compare find out the first different packet that means the following packet almost is different. we needn't do a lot of checkpoint in this time, so we set the no-need-checkpoint peroid, default just set 3 second. Signed-off-by: Zhang Chen --- net/colo-compare.c | 13 ++++++++++++- 1 file changed, 12 insertions(+), 1 deletion(-) diff --git a/net/colo-compare.c b/net/colo-compare.c index 6d500e1..0f8e198 100644 --- a/net/colo-compare.c +++ b/net/colo-compare.c @@ -40,6 +40,9 @@ /* TODO: Should be configurable */ #define REGULAR_PACKET_CHECK_MS 3000 =20 +/* TODO: Should be configurable */ +#define CHECKPOINT_MIN_TIME 3000 + /* + CompareState ++ | | @@ -455,6 +458,7 @@ static void colo_compare_connection(void *opaque, void = *user_data) Packet *pkt =3D NULL; GList *result =3D NULL; int ret; + static int64_t checkpoint_time_ms; =20 while (!g_queue_is_empty(&conn->primary_list) && !g_queue_is_empty(&conn->secondary_list)) { @@ -494,7 +498,14 @@ static void colo_compare_connection(void *opaque, void= *user_data) */ trace_colo_compare_main("packet different"); g_queue_push_tail(&conn->primary_list, pkt); - /* TODO: colo_notify_checkpoint();*/ + + if (pkt->creation_ms - checkpoint_time_ms > CHECKPOINT_MIN_TIM= E) { + /* + * TODO: Notify colo frame to do checkpoint. + * colo_compare_inconsistent_notify(); + */ + checkpoint_time_ms =3D pkt->creation_ms; + } break; } } --=20 2.7.4 From nobody Sat May 4 08:14:50 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1499925304593274.74862839255206; Wed, 12 Jul 2017 22:55:04 -0700 (PDT) Received: from localhost ([::1]:57301 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dVX5b-0000Za-DI for importer@patchew.org; Thu, 13 Jul 2017 01:55:03 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:36108) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dVX3v-0007O6-FF for qemu-devel@nongnu.org; Thu, 13 Jul 2017 01:53:20 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1dVX3s-0006G0-CU for qemu-devel@nongnu.org; Thu, 13 Jul 2017 01:53:19 -0400 Received: from [59.151.112.132] (port=25092 helo=heian.cn.fujitsu.com) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dVX3r-0006C3-Vt for qemu-devel@nongnu.org; Thu, 13 Jul 2017 01:53:16 -0400 Received: from unknown (HELO cn.fujitsu.com) ([10.167.33.5]) by heian.cn.fujitsu.com with ESMTP; 13 Jul 2017 13:53:04 +0800 Received: from G08CNEXCHPEKD02.g08.fujitsu.local (unknown [10.167.33.83]) by cn.fujitsu.com (Postfix) with ESMTP id D70BE46B5246; Thu, 13 Jul 2017 13:53:03 +0800 (CST) Received: from localhost.localdomain (10.167.226.56) by G08CNEXCHPEKD02.g08.fujitsu.local (10.167.33.89) with Microsoft SMTP Server (TLS) id 14.3.319.2; Thu, 13 Jul 2017 13:53:01 +0800 X-IronPort-AV: E=Sophos;i="5.22,518,1449504000"; d="scan'208";a="21231269" From: Zhang Chen To: qemu devel , Jason Wang Date: Thu, 13 Jul 2017 13:52:53 +0800 Message-ID: <1499925175-21218-3-git-send-email-zhangchen.fnst@cn.fujitsu.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1499925175-21218-1-git-send-email-zhangchen.fnst@cn.fujitsu.com> References: <1499925175-21218-1-git-send-email-zhangchen.fnst@cn.fujitsu.com> MIME-Version: 1.0 X-Originating-IP: [10.167.226.56] X-yoursite-MailScanner-ID: D70BE46B5246.ACF88 X-yoursite-MailScanner: Found to be clean X-yoursite-MailScanner-From: zhangchen.fnst@cn.fujitsu.com X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 59.151.112.132 Subject: [Qemu-devel] [PATCH V2 2/4] net/colo-compare.c: Compare the tcp packets that has the same sequence number X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Li Zhijian , zhanghailiang , Zhang Chen Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" If primary packet's sequence number not same with secondary packet's sequence number, no need to compare the packet other field. Signed-off-by: Zhang Chen --- net/colo-compare.c | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/net/colo-compare.c b/net/colo-compare.c index 0f8e198..2caeb80 100644 --- a/net/colo-compare.c +++ b/net/colo-compare.c @@ -222,6 +222,12 @@ static int colo_packet_compare_tcp(Packet *spkt, Packe= t *ppkt) ptcp =3D (struct tcphdr *)ppkt->transport_header; stcp =3D (struct tcphdr *)spkt->transport_header; =20 + if ((ptcp->th_flags & TH_SYN) !=3D TH_SYN && + ptcp->th_seq !=3D stcp->th_seq) { + trace_colo_compare_main("colo_packet_compare_tcp seq not same"); + return -1; + } + /* * The 'identification' field in the IP header is *very* random * it almost never matches. Fudge this by ignoring differences in --=20 2.7.4 From nobody Sat May 4 08:14:50 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1499925406945259.4055761437211; Wed, 12 Jul 2017 22:56:46 -0700 (PDT) Received: from localhost ([::1]:57310 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dVX7F-0001vG-SA for importer@patchew.org; Thu, 13 Jul 2017 01:56:45 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:36113) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dVX3v-0007O7-Nh for qemu-devel@nongnu.org; Thu, 13 Jul 2017 01:53:20 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1dVX3u-0006Is-SJ for qemu-devel@nongnu.org; Thu, 13 Jul 2017 01:53:19 -0400 Received: from [59.151.112.132] (port=31473 helo=heian.cn.fujitsu.com) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dVX3t-0006Gz-Ud for qemu-devel@nongnu.org; Thu, 13 Jul 2017 01:53:18 -0400 Received: from unknown (HELO cn.fujitsu.com) ([10.167.33.5]) by heian.cn.fujitsu.com with ESMTP; 13 Jul 2017 13:53:04 +0800 Received: from G08CNEXCHPEKD02.g08.fujitsu.local (unknown [10.167.33.83]) by cn.fujitsu.com (Postfix) with ESMTP id 013F146B5249; Thu, 13 Jul 2017 13:53:05 +0800 (CST) Received: from localhost.localdomain (10.167.226.56) by G08CNEXCHPEKD02.g08.fujitsu.local (10.167.33.89) with Microsoft SMTP Server (TLS) id 14.3.319.2; Thu, 13 Jul 2017 13:53:03 +0800 X-IronPort-AV: E=Sophos;i="5.22,518,1449504000"; d="scan'208";a="21231272" From: Zhang Chen To: qemu devel , Jason Wang Date: Thu, 13 Jul 2017 13:52:54 +0800 Message-ID: <1499925175-21218-4-git-send-email-zhangchen.fnst@cn.fujitsu.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1499925175-21218-1-git-send-email-zhangchen.fnst@cn.fujitsu.com> References: <1499925175-21218-1-git-send-email-zhangchen.fnst@cn.fujitsu.com> MIME-Version: 1.0 X-Originating-IP: [10.167.226.56] X-yoursite-MailScanner-ID: 013F146B5249.AC066 X-yoursite-MailScanner: Found to be clean X-yoursite-MailScanner-From: zhangchen.fnst@cn.fujitsu.com X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 59.151.112.132 Subject: [Qemu-devel] [PATCH V2 3/4] net/colo-compare.c: Optimize unpredictable tcp options comparison X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Li Zhijian , zhanghailiang , Zhang Chen Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" When network is busy, some tcp options(like sack) will unpredictable occur in primary side or secondary side. it will make packet size not same, but the two packet's payload is identical. colo just care about packet payload, so we skip the option field. Signed-off-by: Zhang Chen --- net/colo-compare.c | 34 +++++++++++++++++++++++++--------- 1 file changed, 25 insertions(+), 9 deletions(-) diff --git a/net/colo-compare.c b/net/colo-compare.c index 2caeb80..6406c4a 100644 --- a/net/colo-compare.c +++ b/net/colo-compare.c @@ -183,7 +183,10 @@ static int packet_enqueue(CompareState *s, int mode) * return: 0 means packet same * > 0 || < 0 means packet different */ -static int colo_packet_compare_common(Packet *ppkt, Packet *spkt, int offs= et) +static int colo_packet_compare_common(Packet *ppkt, + Packet *spkt, + int poffset, + int soffset) { if (trace_event_get_state(TRACE_COLO_COMPARE_MISCOMPARE)) { char pri_ip_src[20], pri_ip_dst[20], sec_ip_src[20], sec_ip_dst[20= ]; @@ -198,9 +201,10 @@ static int colo_packet_compare_common(Packet *ppkt, Pa= cket *spkt, int offset) sec_ip_src, sec_ip_dst); } =20 - if (ppkt->size =3D=3D spkt->size) { - return memcmp(ppkt->data + offset, spkt->data + offset, - spkt->size - offset); + if (ppkt->size =3D=3D spkt->size || poffset !=3D soffset) { + return memcmp(ppkt->data + poffset, + spkt->data + soffset, + spkt->size - soffset); } else { trace_colo_compare_main("Net packet size are not the same"); return -1; @@ -263,12 +267,22 @@ static int colo_packet_compare_tcp(Packet *spkt, Pack= et *ppkt) * so we just need skip this field. */ if (ptcp->th_off > 5) { - ptrdiff_t tcp_offset; - tcp_offset =3D ppkt->transport_header - (uint8_t *)ppkt->data + ptrdiff_t ptcp_offset, stcp_offset; + + ptcp_offset =3D ppkt->transport_header - (uint8_t *)ppkt->data + (ptcp->th_off * 4); - res =3D colo_packet_compare_common(ppkt, spkt, tcp_offset); + stcp_offset =3D spkt->transport_header - (uint8_t *)spkt->data + + (stcp->th_off * 4); + + /* + * When network is busy, some tcp options(like sack) will unpredic= table + * occur in primary side or secondary side. it will make packet si= ze + * not same, but the two packet's payload is identical. colo just + * care about packet payload, so we skip the option field. + */ + res =3D colo_packet_compare_common(ppkt, spkt, ptcp_offset, stcp_o= ffset); } else if (ptcp->th_sum =3D=3D stcp->th_sum) { - res =3D colo_packet_compare_common(ppkt, spkt, ETH_HLEN); + res =3D colo_packet_compare_common(ppkt, spkt, ETH_HLEN, ETH_HLEN); } else { res =3D -1; } @@ -328,6 +342,7 @@ static int colo_packet_compare_udp(Packet *spkt, Packet= *ppkt) * the ip payload here. */ ret =3D colo_packet_compare_common(ppkt, spkt, + network_header_length + ETH_HLEN, network_header_length + ETH_HLEN); =20 if (ret) { @@ -365,6 +380,7 @@ static int colo_packet_compare_icmp(Packet *spkt, Packe= t *ppkt) * the ip payload here. */ if (colo_packet_compare_common(ppkt, spkt, + network_header_length + ETH_HLEN, network_header_length + ETH_HLEN)) { trace_colo_compare_icmp_miscompare("primary pkt size", ppkt->size); @@ -402,7 +418,7 @@ static int colo_packet_compare_other(Packet *spkt, Pack= et *ppkt) sec_ip_src, sec_ip_dst); } =20 - return colo_packet_compare_common(ppkt, spkt, 0); + return colo_packet_compare_common(ppkt, spkt, 0, 0); } =20 static int colo_old_packet_check_one(Packet *pkt, int64_t *check_time) --=20 2.7.4 From nobody Sat May 4 08:14:50 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1499925407468126.9515504057025; Wed, 12 Jul 2017 22:56:47 -0700 (PDT) Received: from localhost ([::1]:57311 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dVX7F-0001vR-8e for importer@patchew.org; Thu, 13 Jul 2017 01:56:45 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:36152) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dVX3x-0007OP-DO for qemu-devel@nongnu.org; Thu, 13 Jul 2017 01:53:22 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1dVX3w-0006Jw-7s for qemu-devel@nongnu.org; Thu, 13 Jul 2017 01:53:21 -0400 Received: from [59.151.112.132] (port=10613 helo=heian.cn.fujitsu.com) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dVX3v-0006IE-3Y for qemu-devel@nongnu.org; Thu, 13 Jul 2017 01:53:20 -0400 Received: from unknown (HELO cn.fujitsu.com) ([10.167.33.5]) by heian.cn.fujitsu.com with ESMTP; 13 Jul 2017 13:53:10 +0800 Received: from G08CNEXCHPEKD02.g08.fujitsu.local (unknown [10.167.33.83]) by cn.fujitsu.com (Postfix) with ESMTP id 0ED0A46B4C97; Thu, 13 Jul 2017 13:53:06 +0800 (CST) Received: from localhost.localdomain (10.167.226.56) by G08CNEXCHPEKD02.g08.fujitsu.local (10.167.33.89) with Microsoft SMTP Server (TLS) id 14.3.319.2; Thu, 13 Jul 2017 13:53:04 +0800 X-IronPort-AV: E=Sophos;i="5.22,518,1449504000"; d="scan'208";a="21231275" From: Zhang Chen To: qemu devel , Jason Wang Date: Thu, 13 Jul 2017 13:52:55 +0800 Message-ID: <1499925175-21218-5-git-send-email-zhangchen.fnst@cn.fujitsu.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1499925175-21218-1-git-send-email-zhangchen.fnst@cn.fujitsu.com> References: <1499925175-21218-1-git-send-email-zhangchen.fnst@cn.fujitsu.com> MIME-Version: 1.0 X-Originating-IP: [10.167.226.56] X-yoursite-MailScanner-ID: 0ED0A46B4C97.AE0DF X-yoursite-MailScanner: Found to be clean X-yoursite-MailScanner-From: zhangchen.fnst@cn.fujitsu.com X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 59.151.112.132 Subject: [Qemu-devel] [PATCH V2 4/4] net/colo-compare.c: Adjust net queue pop order for performance X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Li Zhijian , zhanghailiang , Zhang Chen Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" The packet_enqueue() use g_queue_push_tail() to enqueue net packet, so it is more efficent way use g_queue_pop_head() to get packet for compare. That will improve the success rate of comparison. In my test the performance of ftp put 1000M file will increase 10% Signed-off-by: Zhang Chen --- net/colo-compare.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/net/colo-compare.c b/net/colo-compare.c index 6406c4a..9397269 100644 --- a/net/colo-compare.c +++ b/net/colo-compare.c @@ -484,7 +484,7 @@ static void colo_compare_connection(void *opaque, void = *user_data) =20 while (!g_queue_is_empty(&conn->primary_list) && !g_queue_is_empty(&conn->secondary_list)) { - pkt =3D g_queue_pop_tail(&conn->primary_list); + pkt =3D g_queue_pop_head(&conn->primary_list); switch (conn->ip_proto) { case IPPROTO_TCP: result =3D g_queue_find_custom(&conn->secondary_list, @@ -519,7 +519,7 @@ static void colo_compare_connection(void *opaque, void = *user_data) * until next comparison. */ trace_colo_compare_main("packet different"); - g_queue_push_tail(&conn->primary_list, pkt); + g_queue_push_head(&conn->primary_list, pkt); =20 if (pkt->creation_ms - checkpoint_time_ms > CHECKPOINT_MIN_TIM= E) { /* --=20 2.7.4