From nobody Sun Sep 14 06:32:20 2025 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass(p=reject dis=none) header.from=cloud.com ARC-Seal: i=1; a=rsa-sha256; t=1757671105; cv=none; d=zohomail.com; s=zohoarc; b=S9GG8wC28HGHd5Y+a4l81G/Suv6LnNVeBzmtOmNwHXVeE1J+5XKaT//0L/j9r/YIAcdszZPUQZfXF5bDFInDpvemqN+XeVGYSzpuUVh90UprIW3/camJ2OJtzn6eK0w/3qf8DCkx/LrNiPjZ3hlVtHrEHzGggVYhy1YL5EqUBNQ= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1757671105; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=40hmlr5BYX9rZsXVh4CU9sX4LFOJM9bVuQanQklyxpE=; b=hioSCk1uM0iAnGNqxwBYK/NaYPWZLD/RDhKoSrb0smLExfYUBLz+mp2Zay+FUZCE8XqpNemh/LF/HkbUBBfr/Bs/fC5PBLAl0v5jndlaLTtcCZ0JwrIcMwVrO7sJdWhNHWzt+Bgu2evhGJl5WhbTQ6sIaNdSA9+zdDZx7sSpbqw= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass header.from= (p=reject dis=none) Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1757671105304680.8004408087754; Fri, 12 Sep 2025 02:58:25 -0700 (PDT) Received: from list by lists.xenproject.org with outflank-mailman.1121747.1465858 (Exim 4.92) (envelope-from ) id 1ux0XX-0005CT-9W; Fri, 12 Sep 2025 09:57:59 +0000 Received: by outflank-mailman (output) from mailman id 1121747.1465858; Fri, 12 Sep 2025 09:57:59 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1ux0XX-0005CM-6W; Fri, 12 Sep 2025 09:57:59 +0000 Received: by outflank-mailman (input) for mailman id 1121747; Fri, 12 Sep 2025 09:57:57 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1ux0XV-0005CG-SY for xen-devel@lists.xenproject.org; Fri, 12 Sep 2025 09:57:57 +0000 Received: from mail-wm1-x334.google.com (mail-wm1-x334.google.com [2a00:1450:4864:20::334]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS id f4e0cc32-8fbe-11f0-9d13-b5c5bf9af7f9; Fri, 12 Sep 2025 11:57:56 +0200 (CEST) Received: by mail-wm1-x334.google.com with SMTP id 5b1f17b1804b1-45cb5492350so12733345e9.1 for ; Fri, 12 Sep 2025 02:57:56 -0700 (PDT) Received: from localhost.localdomain ([87.114.69.104]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-3e7607cd9fasm5850043f8f.35.2025.09.12.02.57.54 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 12 Sep 2025 02:57:55 -0700 (PDT) X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: f4e0cc32-8fbe-11f0-9d13-b5c5bf9af7f9 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cloud.com; s=cloud; t=1757671076; x=1758275876; darn=lists.xenproject.org; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:from:to:cc:subject:date:message-id:reply-to; bh=40hmlr5BYX9rZsXVh4CU9sX4LFOJM9bVuQanQklyxpE=; b=CwYOARX8LACft4Qw6D8QzzVOK1kK4w/G2SSAMwySKvd8NSUMRS2aHpeDxd9PJDky+g guM1vSVSWgKenHsh+sA0GqJoqx1j0dD0kG0uAZm2BFIc/tlncYehW9qvQ5hI3wnY9ZwL 7H7ukna5OEqi7l0nZ+blzbC4ZeUPmJ4ks63PE= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1757671076; x=1758275876; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=40hmlr5BYX9rZsXVh4CU9sX4LFOJM9bVuQanQklyxpE=; b=obY9VoKrIMSLikge/32/hm984ID6NCoRJ+hgwE4JMbIhbxEWFYM6lpHeMO5d4dCEO8 L8DGCthG+2GRx99DPAwlR4uL0Ifw+RvFDF3eNgj2Dd5AxmrE+6hZFgPEk/DvCjwHUCGP WKctI2+QYJWn1eg7qelJ6n+6Nbjc8pHwctaqV9BVXqQ/2CNvptanE3LLh5g1uYO96TY6 FdEQpd+8MQoBz2X0xzMsgIn85alO1aadJNsjP3Gc/mL6L7gHHCKGbnSMzMqYhiy+EKIa IL9XmK+eoBxduf9CXZ1GCgIuhn77cEW8Oo+aUdVbX5sIYWka/uSG19CQh7ag+l1m7IQo Bnag== X-Gm-Message-State: AOJu0YwViN1lbRjbsK3vTEqHRsK8hmeTum8slvhufOr0oDx6MtOrDFj/ 4EIdoLcSR18bsUfW/Q2jPYGfqnt/Td+6AAc6jllQ2sXxtAzb5/K6NeuItOHua5CLPIwFCgw3y0k wKM16LKY= X-Gm-Gg: ASbGncsCJuMbsVqiLqKRJhHsZ12VCCNk5iQKPTrWhm7b/SHiNiQSc8bhy7aTKzDQA44 +xJzC2firrDPd+T7JL60i6SopK9kJEy6qZdMy+c3yfm08RY72nXltol4OXUcVY6CzC763zkj/vq ZtTbIa+uW8YyV9vOEaxIfQbnGWIBuo6taqT+v7gSie0JtF6jtG9HIayXaC+N+kS7nb9C7qXQUv/ MrO+fBriYIbhw7UmW52GfbmkXjAHBW5nS3341nLCzUaIRcCYWK4nJFeTJxrDHqX32DhST2pwd8N jCHOL8HLAA38fLLQkPKXJ0j6z5Ny+iEYhaEICGSVO1IHVScrTeZOJg2mXHchAOEh7tLdkmUHM3V nYpkMSQNs/CoQUivNUbx1zhrp9V7rhx0YGtXcv/kTSfzQ2gErfqG6 X-Google-Smtp-Source: AGHT+IGs+stwR7Er4cFYsKF0o4kRKljj4gxRt6JfpX/6GkjzcAiePSBMlTp/aPec5ahAwYotQImiiQ== X-Received: by 2002:a05:600c:6812:b0:45d:98be:ee91 with SMTP id 5b1f17b1804b1-45f211d4f1emr24183805e9.9.1757671075611; Fri, 12 Sep 2025 02:57:55 -0700 (PDT) From: Frediano Ziglio To: xen-devel@lists.xenproject.org Cc: Frediano Ziglio , Anthony PERARD , Juergen Gross , =?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= , Andrew Cooper , Oleksii Kurochko Subject: [PATCH v3] tools/libs: Use superpages where possible on migrate/resume Date: Fri, 12 Sep 2025 10:57:44 +0100 Message-ID: <20250912095744.99181-1-frediano.ziglio@cloud.com> X-Mailer: git-send-email 2.43.0 MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @cloud.com) X-ZM-MESSAGEID: 1757671107206116600 Content-Type: text/plain; charset="utf-8" Try to allocate larger order pages. With some test memory program stressing TLB (many small random memory accesses) you can get 15% performance improves. On the first memory iteration the sender is currently sending memory in 4mb aligned chunks which allows the receiver to allocate most pages as 2mb superpages instead of single 4kb pages. This works even for HVM where the first 2mb contains some holes. This change does not handle 1gb superpages as this will require change in the protocol to preallocate space. Signed-off-by: Frediano Ziglio Release-Acked-by: Oleksii Kurochko Reviewed-by: Andrew Cooper --- Changes since v1: - updated commit message and subject; - change the implementation detecting possible 2mb pages inside the packet sent allowing more 2mb superpages. Changes since v2: - change implementation simplifying detecting and allocations of 2mb pages. --- tools/libs/guest/xg_sr_restore.c | 45 +++++++++++++++++++++++++++++--- 1 file changed, 42 insertions(+), 3 deletions(-) diff --git a/tools/libs/guest/xg_sr_restore.c b/tools/libs/guest/xg_sr_rest= ore.c index 06231ca826..ea5a137612 100644 --- a/tools/libs/guest/xg_sr_restore.c +++ b/tools/libs/guest/xg_sr_restore.c @@ -129,6 +129,30 @@ static int pfn_set_populated(struct xc_sr_context *ctx= , xen_pfn_t pfn) return 0; } =20 +#if defined(__i386__) || defined(__x86_64__) +/* Order of the smallest superpage */ +#define SMALL_SUPERPAGE_ORDER 9 +#else +#error Define SMALL_SUPERPAGE_ORDER for this platform +#endif + +static bool populate_small_superpage(struct xc_sr_context *ctx, xen_pfn_t = pfn) +{ + xen_pfn_t mfn =3D pfn; + + if ( xc_domain_populate_physmap_exact( + ctx->xch, ctx->domid, 1, SMALL_SUPERPAGE_ORDER, 0, &mfn) ) + return false; + + if ( mfn =3D=3D INVALID_MFN ) + return false; + + for ( size_t i =3D 0; i < (1 << SMALL_SUPERPAGE_ORDER); ++i ) + ctx->restore.ops.set_gfn(ctx, pfn + i, mfn + i); + + return true; +} + /* * Given a set of pfns, obtain memory from Xen to fill the physmap for the * unpopulated subset. If types is NULL, no page type checking is perform= ed @@ -142,6 +166,9 @@ int populate_pfns(struct xc_sr_context *ctx, unsigned i= nt count, *pfns =3D malloc(count * sizeof(*pfns)); unsigned int i, nr_pfns =3D 0; int rc =3D -1; + xen_pfn_t prev =3D 0; + unsigned num_contiguous =3D 0; + xen_pfn_t mask =3D ~((~(xen_pfn_t)0) << SMALL_SUPERPAGE_ORDER); =20 if ( !mfns || !pfns ) { @@ -152,14 +179,26 @@ int populate_pfns(struct xc_sr_context *ctx, unsigned= int count, =20 for ( i =3D 0; i < count; ++i ) { + xen_pfn_t pfn =3D original_pfns[i]; + if ( (!types || page_type_to_populate(types[i])) && - !pfn_is_populated(ctx, original_pfns[i]) ) + !pfn_is_populated(ctx, pfn) ) { - rc =3D pfn_set_populated(ctx, original_pfns[i]); + rc =3D pfn_set_populated(ctx, pfn); if ( rc ) goto err; - pfns[nr_pfns] =3D mfns[nr_pfns] =3D original_pfns[i]; + pfns[nr_pfns] =3D mfns[nr_pfns] =3D pfn; ++nr_pfns; + if ( pfn !=3D prev + 1 ) + num_contiguous =3D 0; + num_contiguous++; + prev =3D pfn; + if ( num_contiguous > mask && (pfn & mask) =3D=3D mask && + populate_small_superpage(ctx, pfn - mask) ) + { + nr_pfns -=3D mask + 1; + num_contiguous =3D 0; + } } } =20 --=20 2.43.0