From nobody Thu Oct 30 22:53:01 2025 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass(p=reject dis=none) header.from=cloud.com ARC-Seal: i=1; a=rsa-sha256; t=1756810616; cv=none; d=zohomail.com; s=zohoarc; b=TcbKsHIw//alQhV9Nmtt7dnIQebWy0OuurjjY623BQIoOIlEvOCYqxD8j4v5v7kSWIF1Cvv6qu+2K/IbkZs0HuTGnpmJc3ILFX7RfcHLNrLX449yytb++yo+CBRcv+gDYIRMkyje5Y1KUHDDKZb40dSx3rVLolPeuVbqwfPZvlA= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1756810616; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=uZi8Sj1r6fD/XAgHc/5d4qkPiGFTtkfD49OTXdflNVI=; b=bPXdHvrVO/UJvCFPK7RBc8InfylPZKALVz7k7n5Jw8YKVIPn0xjpK7tZc+PLZVl1yQRSinoBXAXjHRt/QrW9Q3pxOfIBP84UTONJXwJG5utlXbDWBnt+eMwqWMoBwoT5jLOn8jaSF3VC8wTHJn99huHpFugFaH6tiLvYNftZD3I= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass header.from= (p=reject dis=none) Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 175681061693857.75775332511125; Tue, 2 Sep 2025 03:56:56 -0700 (PDT) Received: from list by lists.xenproject.org with outflank-mailman.1106065.1456851 (Exim 4.92) (envelope-from ) id 1utOgl-0006KK-C7; Tue, 02 Sep 2025 10:56:35 +0000 Received: by outflank-mailman (output) from mailman id 1106065.1456851; Tue, 02 Sep 2025 10:56:35 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1utOgl-0006KD-9X; Tue, 02 Sep 2025 10:56:35 +0000 Received: by outflank-mailman (input) for mailman id 1106065; Tue, 02 Sep 2025 10:56:34 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1utOgk-0006K7-Ck for xen-devel@lists.xenproject.org; Tue, 02 Sep 2025 10:56:34 +0000 Received: from mail-wm1-x32e.google.com (mail-wm1-x32e.google.com [2a00:1450:4864:20::32e]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS id 7cd13767-87eb-11f0-8adc-4578a1afcccb; Tue, 02 Sep 2025 12:56:33 +0200 (CEST) Received: by mail-wm1-x32e.google.com with SMTP id 5b1f17b1804b1-45b84367affso27552735e9.3 for ; Tue, 02 Sep 2025 03:56:33 -0700 (PDT) Received: from localhost.localdomain ([87.114.69.104]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-45b7e74b72esm197226915e9.0.2025.09.02.03.56.31 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 02 Sep 2025 03:56:31 -0700 (PDT) X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 7cd13767-87eb-11f0-8adc-4578a1afcccb DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cloud.com; s=cloud; t=1756810592; x=1757415392; darn=lists.xenproject.org; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:from:to:cc:subject:date:message-id:reply-to; bh=uZi8Sj1r6fD/XAgHc/5d4qkPiGFTtkfD49OTXdflNVI=; b=WSZ5i9hG791qn9o3kz/znb4ZJ+Z0jpdpei/Qy/w/xT26U58jjZ3IVzlUP/MQLEYJCC 5CdiQrZXSiWMhrvrC5AfYogAxsJUPjGK0RWeMVDPqGbM9ggEMizIRwpWkeOtFWwOpx15 Pfr3NwH1f/tgU1BcvFoQB2+jS0sNdgHGXLyFQ= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1756810592; x=1757415392; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=uZi8Sj1r6fD/XAgHc/5d4qkPiGFTtkfD49OTXdflNVI=; b=qyH7BxBWLYMmlZUAOJoQfk/Q/Hb7n2tWKOLCahvyyz75R7NcZo1OzZQCacQk92wn8Z 5jf/8kw2B0iXKmV1R8/DsRYbio6S0Uf/FTgLYMoVbCDvHWQ6yQ/GyePLpgYJ4XhODvsA n/RnXuMxxvGfwT2mbcAosVCdcztB6dRVb7YFH8WFgX2RS6oPVlsmWL4GGcrn7SwlTe2C IKNrXQ79q3tZ0b/JUscdH37X5KLD+EV2bXzhPqq20n6fLHLi80gboBfAYb2w6BpYTlvb 2uttUKgIX45TSzs90I4FzoadLCXm96HMP14M7/41n6hkXhePeRpLfrl4cEoLMKyCXuaD Dcuw== X-Gm-Message-State: AOJu0Yz5bA8/pMtGJ0YqZ7m3/lv96v0hBdQUiFkr1VLi6AvT4EprpS6Y PMjBh2u5e+CGaU2YDZ4uqzaLAhidyPepj7to6yhYjZ845j0ptl0KJmFppYqBivs5QZkikV6C4SB 1IRFDl4Q= X-Gm-Gg: ASbGncsIWOZ792vJr9YIaG5lX+U1jwtoJPiwpG11GeRNL9je2GLpKpJXkE6fzIHXkek hXwkLvWJlwKGY9cJkqA+Wz+wcAOut6XU+zKX8sceGSY7N8iAcyT7AVKJC/KCXVOlhT/9+pMzc7I rWchQ1Xp/uVjZyBqsXKPdcjOJfPjrpHnfRofagc/HMqQhR9VAJSSM/hh8w5LmeOgZB0xgqFpXuP eNGn7T6hsbGGR0GDPph2jQKcethw2yRMLTFSOOL5pWKjTAq/+u1cGwCwnQErmF5i+xsbnoVxy1u RitUaU7Fh5Cs7MqKiH70I8cKioZvEn+Jv7On4q36FwJ7uTA2tCjzGxgNkdjf17RXQYqILYml9J8 BsjLxjxJ/RkzLcQZs2OSAvV2o0pTae2iDu+uxVkq6BA== X-Google-Smtp-Source: AGHT+IGBRNsdWIEVq2lvABTbUUECIfsmVa+EHXBbyet6RJA7rgFMhqYNN2EUThPsqZySlFDKzug8IQ== X-Received: by 2002:a05:600c:1f85:b0:45b:5f99:191c with SMTP id 5b1f17b1804b1-45b8555ca8dmr86401375e9.12.1756810592060; Tue, 02 Sep 2025 03:56:32 -0700 (PDT) From: Frediano Ziglio To: xen-devel@lists.xenproject.org Cc: Frediano Ziglio , Anthony PERARD , Juergen Gross , =?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= , Andrew Cooper Subject: [PATCH v2] tools/libs: Use superpages where possible on migrate/resume Date: Tue, 2 Sep 2025 11:56:25 +0100 Message-ID: <20250902105625.28552-1-frediano.ziglio@cloud.com> X-Mailer: git-send-email 2.43.0 MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @cloud.com) X-ZM-MESSAGEID: 1756810617752116600 Content-Type: text/plain; charset="utf-8" Try to allocate larger order pages. With some test memory program stressing TLB (many small random memory accesses) you can get 15% performance improves. On the first memory iteration the sender is currently sending memory in 4mb aligned chunks which allows the receiver to allocate most pages as 2mb superpages instead of single 4kb pages. This works even for HVM where the first 2mb contains some holes. This change does not handle 1gb superpages as this will require change in the protocol to preallocate space. Signed-off-by: Frediano Ziglio --- Changes since v1: - updated commit message and subject; - change the implementation detecting possible 2mb pages inside the packet sent allowing more 2mb superpages. --- tools/libs/guest/xg_sr_restore.c | 77 ++++++++++++++++++++++++++++++++ 1 file changed, 77 insertions(+) diff --git a/tools/libs/guest/xg_sr_restore.c b/tools/libs/guest/xg_sr_rest= ore.c index 06231ca826..f2018299a7 100644 --- a/tools/libs/guest/xg_sr_restore.c +++ b/tools/libs/guest/xg_sr_restore.c @@ -129,6 +129,80 @@ static int pfn_set_populated(struct xc_sr_context *ctx= , xen_pfn_t pfn) return 0; } =20 +#if defined(__i386__) || defined(__x86_64__) +/* Order of the smallest superpage */ +#define SMALL_SUPERPAGE_ORDER 9 +#else +#error Define SMALL_SUPERPAGE_ORDER for this platform +#endif + +static unsigned int populate_order(struct xc_sr_context *ctx, + unsigned int original_count, + xen_pfn_t *pfns, xen_pfn_t *mfns, + int order) +{ + size_t i =3D original_count, num_superpages; + xen_pfn_t prev =3D 0, order_mask =3D ~((~(xen_pfn_t)0) << order); + xen_pfn_t *const indexes_end =3D mfns + original_count; + xen_pfn_t *indexes =3D indexes_end; + unsigned int count =3D 0; + + while ( i > 0 ) + { + --i; + ++count; + if ( pfns[i] !=3D prev - 1 ) + count =3D 1; + + /* + * Is this the start of a contiguous and aligned number + * of pages ? + */ + if ( (pfns[i] & order_mask) =3D=3D 0 && count > order_mask ) + *--indexes =3D i; + + prev =3D pfns[i]; + } + + count =3D original_count; + + /* No superpages found */ + if ( indexes =3D=3D indexes_end ) + return count; + num_superpages =3D indexes_end - indexes; + + /* Build list of PFNs that will be updated with MFNs */ + mfns =3D indexes - num_superpages; + for ( i =3D 0; i < num_superpages; ++i ) + mfns[i] =3D pfns[indexes[i]]; + + /* Try to allocate, fallback to single pages */ + if ( xc_domain_populate_physmap_exact( + ctx->xch, ctx->domid, num_superpages, order, 0, mfns) ) + return count; + + /* Scan all MFNs allocated */ + for ( i =3D 0; i < num_superpages; ++i ) + { + const xen_pfn_t mfn =3D mfns[i]; + const xen_pfn_t pfn =3D pfns[indexes[i]]; + + /* Check valid */ + if ( mfn =3D=3D INVALID_MFN ) + continue; + + /* Update PFNs using callback */ + for ( size_t j =3D 0; j <=3D order_mask; ++j ) + ctx->restore.ops.set_gfn(ctx, pfn + j, mfn + j); + + /* remove from 4kb pages list */ + count -=3D order_mask + 1; + memmove(pfns + indexes[i], pfns + indexes[i] + order_mask + 1, + sizeof(*pfns) * (count - indexes[i])); + } + return count; +} + /* * Given a set of pfns, obtain memory from Xen to fill the physmap for the * unpopulated subset. If types is NULL, no page type checking is perform= ed @@ -163,6 +237,9 @@ int populate_pfns(struct xc_sr_context *ctx, unsigned i= nt count, } } =20 + /* try optimizing using larger order */ + nr_pfns =3D populate_order(ctx, nr_pfns, pfns, mfns, SMALL_SUPERPAGE_O= RDER); + if ( nr_pfns ) { rc =3D xc_domain_populate_physmap_exact( --=20 2.43.0