From nobody Sat Apr 27 09:25:08 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 164258032376526.867076788628083; Wed, 19 Jan 2022 00:18:43 -0800 (PST) Received: from localhost ([::1]:34380 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nA6Am-0001Qy-VT for importer@patchew.org; Wed, 19 Jan 2022 03:18:31 -0500 Received: from eggs.gnu.org ([209.51.188.92]:58622) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nA62T-0006xB-Ot for qemu-devel@nongnu.org; Wed, 19 Jan 2022 03:09:55 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]:28134) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nA62Q-0000tC-LE for qemu-devel@nongnu.org; Wed, 19 Jan 2022 03:09:51 -0500 Received: from mail-wm1-f69.google.com (mail-wm1-f69.google.com [209.85.128.69]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-537-JsNUn9pvPQStKRL1S6cjhQ-1; Wed, 19 Jan 2022 03:09:46 -0500 Received: by mail-wm1-f69.google.com with SMTP id n13-20020a05600c3b8d00b0034979b7e200so789627wms.4 for ; Wed, 19 Jan 2022 00:09:46 -0800 (PST) Received: from localhost.localdomain ([85.203.46.175]) by smtp.gmail.com with ESMTPSA id a20sm4351606wmb.27.2022.01.19.00.09.39 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Wed, 19 Jan 2022 00:09:44 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1642579789; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=PSDx2fKo/UzR+lo/LkmSaHIqGIyIJEzEwY50huIabEg=; b=JVZyKDqWgrZvfriZ5sjm3ZBcBn4Y+zFZAOzduUQrNYvnJyB+EE1zNJ+fIDCJyHNd9ZD5lA gZ7me99hmgFwZ1NYvgb/Pfa23uNZvsJUtiAnkFOjCRQyFN4VMp07TbJARgTD9PUadw7BBl RvrOHVtGEmKNIAYJXo0c53YwmWQtB+U= X-MC-Unique: JsNUn9pvPQStKRL1S6cjhQ-1 X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=PSDx2fKo/UzR+lo/LkmSaHIqGIyIJEzEwY50huIabEg=; b=BdJbafelxNlxisVXS3+BCVu9tIJompaT9lwjdf8JfyViXCvBM6V39WzPMew6a0tZ58 gU910L5/asIbOJ6P2pXS2Xdaao2ebnkVf2HFa8Jal+lmr60rzlo0rkGHnukj2NGQRa6C Jj9fHj/twYhTqshrHtme4KKu63gVmpjcCaValqDFJoc50MVwJDrkcu/SQoWBXoEQJvh+ 7D3A+Wa9amMv/kCrx/BMbif2riqvQb162J3G2clYF0OvzuC7BGKnS9jmtorHQQDaTFX3 E5izau2TvyP12tCdegv4MrTHl9l+jyRJeVQzPRIuyIPM9umSuPXkplncT9ny/l/3S3HF 5PQg== X-Gm-Message-State: AOAM530fP07Lx0FVSLrba8ryZwsl7n5b88UQZXG5NmWWDcb3maYVKTYW qeQOhu3kDYs48npo3y77MF05vFGchg+y08LQV0oMOpwfjKhBRPNXjsqvrcHpxWIJ+KbWjX6sASF pNETTQTkK97GPoBzNVbPenfj4t8EXfTrSu+Ap4GkwAvFjMvFPrm4kdgxZNIPmeAvm X-Received: by 2002:a05:6000:2a3:: with SMTP id l3mr27930627wry.194.1642579784740; Wed, 19 Jan 2022 00:09:44 -0800 (PST) X-Google-Smtp-Source: ABdhPJzPd+iHNYr5kRzcHnL3fDPcT4VfyS9DWZVfH0BHXl083Z3n+zwCmWaTZWoIn51fzyXdlMMkIw== X-Received: by 2002:a05:6000:2a3:: with SMTP id l3mr27930602wry.194.1642579784523; Wed, 19 Jan 2022 00:09:44 -0800 (PST) From: Peter Xu To: qemu-devel@nongnu.org Subject: [PATCH RFC 01/15] migration: No off-by-one for pss->page update in host page size Date: Wed, 19 Jan 2022 16:09:15 +0800 Message-Id: <20220119080929.39485-2-peterx@redhat.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20220119080929.39485-1-peterx@redhat.com> References: <20220119080929.39485-1-peterx@redhat.com> MIME-Version: 1.0 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=peterx@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=peterx@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -34 X-Spam_score: -3.5 X-Spam_bar: --- X-Spam_report: (-3.5 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.7, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Juan Quintela , Kunkun Jiang , "Dr . David Alan Gilbert" , peterx@redhat.com, Leonardo Bras Soares Passos , Keqian Zhu , Andrey Gruzdev Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZM-MESSAGEID: 1642580325650100001 Content-Type: text/plain; charset="utf-8" We used to do off-by-one fixup for pss->page when finished one host huge pa= ge transfer. That seems to be unnecesary at all. Drop it. Cc: Keqian Zhu Cc: Kunkun Jiang Cc: Andrey Gruzdev Signed-off-by: Peter Xu Reviewed-by: Dr. David Alan Gilbert Reviewed-by: Juan Quintela --- migration/ram.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/migration/ram.c b/migration/ram.c index 5234d1ece1..381ad56d26 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -1611,7 +1611,7 @@ static int ram_save_release_protection(RAMState *rs, = PageSearchStatus *pss, /* Check if page is from UFFD-managed region. */ if (pss->block->flags & RAM_UF_WRITEPROTECT) { void *page_address =3D pss->block->host + (start_page << TARGET_PA= GE_BITS); - uint64_t run_length =3D (pss->page - start_page + 1) << TARGET_PAG= E_BITS; + uint64_t run_length =3D (pss->page - start_page) << TARGET_PAGE_BI= TS; =20 /* Flush async buffers before un-protect. */ qemu_fflush(rs->f); @@ -2230,7 +2230,7 @@ static int ram_save_host_page(RAMState *rs, PageSearc= hStatus *pss, offset_in_ramblock(pss->block, ((ram_addr_t)pss->page) << TARGET_PAGE_BIT= S)); /* The offset we leave with is the min boundary of host page and block= */ - pss->page =3D MIN(pss->page, hostpage_boundary) - 1; + pss->page =3D MIN(pss->page, hostpage_boundary); =20 res =3D ram_save_release_protection(rs, pss, start_page); return (res < 0 ? res : pages); --=20 2.32.0 From nobody Sat Apr 27 09:25:08 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1642580776478338.5996498915531; Wed, 19 Jan 2022 00:26:16 -0800 (PST) Received: from localhost ([::1]:43020 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nA6II-0007t2-Kq for importer@patchew.org; Wed, 19 Jan 2022 03:26:14 -0500 Received: from eggs.gnu.org ([209.51.188.92]:58654) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nA62W-0006xF-Ch for qemu-devel@nongnu.org; Wed, 19 Jan 2022 03:09:56 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]:22920) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nA62S-0000tc-Jp for qemu-devel@nongnu.org; Wed, 19 Jan 2022 03:09:54 -0500 Received: from mail-wm1-f69.google.com (mail-wm1-f69.google.com [209.85.128.69]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-349-IuoSFsOvOU2Yf76F8Gxs8w-1; Wed, 19 Jan 2022 03:09:51 -0500 Received: by mail-wm1-f69.google.com with SMTP id c188-20020a1c35c5000000b00346a2160ea8so782436wma.9 for ; Wed, 19 Jan 2022 00:09:50 -0800 (PST) Received: from localhost.localdomain ([85.203.46.175]) by smtp.gmail.com with ESMTPSA id a20sm4351606wmb.27.2022.01.19.00.09.44 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Wed, 19 Jan 2022 00:09:48 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1642579792; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=p5w+DAZKfAMMjbu0RXHKRc9bVXuM3CL/GD3quFyKBLo=; b=OWgHAboTuF72G6TMGiNI8YfyNwXThfEtPH1GWKNMF+x0qSuc2egN9cs+d3K3xbIR2TXDSt U6lRntOQzRGuLBQnRKN9wLwdgJs9GBWDw9527oIM7NcUTFdb/4yHNL5xheOvwrSOjlgaOA ourOHKcJ1DncCsO+XuxtVpeIEfJja6Y= X-MC-Unique: IuoSFsOvOU2Yf76F8Gxs8w-1 X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=p5w+DAZKfAMMjbu0RXHKRc9bVXuM3CL/GD3quFyKBLo=; b=vGCQ1fFvmx2md8nXBZJ3EueJt6FWn3d87moTPVkwRTVZAIBH3ezfP2NmH/3WSgGZ43 9dPU7bZdVyR4lLRlA6pL2OiS0L3HA9UJbuDzkKMj5tdXwx4NQ3jYwAu+PHn0evoyedAW pX1LrqB567sWn6jJq+46GdDqJA1I5rlRyG02S+xKLXow6SnXZhI7nymLes3F1Ec5JPzv Yq+5qCYnD+iR94mZH6zxy3Ka2kCfgeqwOz8P9q1BSZgUvKSsEflT3i/16pj+x1fayLHZ MV9RKayCGYd9rO88+4oHOpi5fDvYHZydOjQSpKny1cPGddCRsu9Iy+COkFxn7d5LOdwZ ve4Q== X-Gm-Message-State: AOAM533sDuIzGbXYod0RmL4K+Jg/xopND6iVv/vpDfdWKSGj7CqfOuoB IcKyGObdvT1AIDEkpvEGOV3kNCzcf0yw/SyVblxvbIKjABNONAiZFPRV8dwD4LDz+QV+TYg6YZ0 nzUbdSw2su7s0o9HARV475bVd+6jmNuU3jiiqXspcQjVPWgeo6NsJrH+GIeOE21z2 X-Received: by 2002:a5d:5590:: with SMTP id i16mr26735084wrv.34.1642579789469; Wed, 19 Jan 2022 00:09:49 -0800 (PST) X-Google-Smtp-Source: ABdhPJyDq09WHGjvr6pY/lTc9mBrSZ0ODNH6EBrgcxeiI1OK1gDbx+zLeK79lgXuW3uWuHUVS9q2mA== X-Received: by 2002:a5d:5590:: with SMTP id i16mr26735057wrv.34.1642579789121; Wed, 19 Jan 2022 00:09:49 -0800 (PST) From: Peter Xu To: qemu-devel@nongnu.org Subject: [PATCH RFC 02/15] migration: Allow pss->page jump over clean pages Date: Wed, 19 Jan 2022 16:09:16 +0800 Message-Id: <20220119080929.39485-3-peterx@redhat.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20220119080929.39485-1-peterx@redhat.com> References: <20220119080929.39485-1-peterx@redhat.com> MIME-Version: 1.0 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=peterx@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=peterx@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -34 X-Spam_score: -3.5 X-Spam_bar: --- X-Spam_report: (-3.5 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.7, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Juan Quintela , Kunkun Jiang , "Dr . David Alan Gilbert" , peterx@redhat.com, Leonardo Bras Soares Passos , Keqian Zhu Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZM-MESSAGEID: 1642580777999100001 Content-Type: text/plain; charset="utf-8" Commit ba1b7c812c ("migration/ram: Optimize ram_save_host_page()") managed = to optimize host huge page use case by scanning the dirty bitmap when looking = for the next dirty small page to migrate. However when updating the pss->page before returning from that function, we used MIN() of these two values: (1) next dirty bit, or (2) end of current s= ent huge page, to fix up pss->page. That sounds unnecessary, because I see nowhere that requires pss->page to be not going over current huge page boundary. What we need here is probably MAX() instead of MIN() so that we'll start scanning from the next dirty bit next time. Since pss->page can't be smaller than hostpage_boundary (the loop guarantees it), it probably means we don't need to fix it up at all. Cc: Keqian Zhu Cc: Kunkun Jiang Signed-off-by: Peter Xu --- migration/ram.c | 2 -- 1 file changed, 2 deletions(-) diff --git a/migration/ram.c b/migration/ram.c index 381ad56d26..94b0ad4234 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -2229,8 +2229,6 @@ static int ram_save_host_page(RAMState *rs, PageSearc= hStatus *pss, } while ((pss->page < hostpage_boundary) && offset_in_ramblock(pss->block, ((ram_addr_t)pss->page) << TARGET_PAGE_BIT= S)); - /* The offset we leave with is the min boundary of host page and block= */ - pss->page =3D MIN(pss->page, hostpage_boundary); =20 res =3D ram_save_release_protection(rs, pss, start_page); return (res < 0 ? res : pages); --=20 2.32.0 From nobody Sat Apr 27 09:25:08 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1642581180145402.23473917612796; Wed, 19 Jan 2022 00:33:00 -0800 (PST) Received: from localhost ([::1]:51100 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nA6Oo-0005G5-8j for importer@patchew.org; Wed, 19 Jan 2022 03:32:58 -0500 Received: from eggs.gnu.org ([209.51.188.92]:58712) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nA62g-00070W-4X for qemu-devel@nongnu.org; Wed, 19 Jan 2022 03:10:06 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:39887) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nA62X-0000vQ-5E for qemu-devel@nongnu.org; Wed, 19 Jan 2022 03:09:58 -0500 Received: from mail-wm1-f70.google.com (mail-wm1-f70.google.com [209.85.128.70]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-322-oxoB_fJLPBq29nvnUm3TOQ-1; Wed, 19 Jan 2022 03:09:55 -0500 Received: by mail-wm1-f70.google.com with SMTP id s17-20020a7bc0d1000000b00348737ba2a2so1339733wmh.2 for ; Wed, 19 Jan 2022 00:09:54 -0800 (PST) Received: from localhost.localdomain ([85.203.46.175]) by smtp.gmail.com with ESMTPSA id a20sm4351606wmb.27.2022.01.19.00.09.49 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Wed, 19 Jan 2022 00:09:52 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1642579796; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=r648Y3X/bfN5BCV7H2OloEH+uVhiyU6Y+C3RrlsatDo=; b=SIRsomhLZn8xlhrR9x4oqO5OJGP1pStLmuW0jrtgXcVkuTLLT1/nmOz9z/BmbWtOhF570+ XHJwN6jNrnf9Q20x/DtbZzIlmip8/s/bGQK0u0gC0365cqCP1xsmDtBH8pRLaDD/Yg+lOo GtBR+ZnaFJTdoBiPjBxPoksD8S2VBI4= X-MC-Unique: oxoB_fJLPBq29nvnUm3TOQ-1 X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=r648Y3X/bfN5BCV7H2OloEH+uVhiyU6Y+C3RrlsatDo=; b=YbpIrdTDyMco7iuJ0vjn0DkbJqwAIVeYSdsP2fxouC+biRt21utn7GCXwyG38A/7T6 wElEmuFEDdJ/CtNxbjeg50iuVnyeOf1PPKzNmNZbQnRkJTLx2O9kyB39fEj1wnSSgEoz ZxO6pp5JnUOX9sZqVZCfTSxTklxxuMdAl+qkZJL58VLZGIE4SOeyo6sw/K7Ycs2/k4va R3yNrVfFys6QmHqgqiwCuVYIP5s/Hne5cZ1iE7W/oAgoguc4rYrqa4p34LHo+j4SZHgX x29NlcWJCG1zIxL8NDyZpkim7URgXCpTfnB5vfwqG+IusGMrzIPVZtzU96MtpOwUEyRE xrZw== X-Gm-Message-State: AOAM530NJsMG9e87Ryzluu8z5/wlbXZlomPBcMdRGCiIBmxbGfpx5aVI IGBvtCH+5BipA1/NNW1/GzKIeAD866QeK2kLxMfU4RBC5poQ8NAiRz0sS8B/DP+0QjCmegTJO/i E9dMF/4zTK17ISjcAACe77wDhAwKlg7rUAUAvMPBwobYh3ERJc395VEXyUsEhcwlG X-Received: by 2002:a05:600c:4f16:: with SMTP id l22mr2304619wmq.122.1642579793325; Wed, 19 Jan 2022 00:09:53 -0800 (PST) X-Google-Smtp-Source: ABdhPJwvbuRGHw8shqHx/kqnSXogOiMfHZVnDrneMu9PayetmUffFh0sw55Dfe6pkR7v+NMYFNvcZg== X-Received: by 2002:a05:600c:4f16:: with SMTP id l22mr2304594wmq.122.1642579793046; Wed, 19 Jan 2022 00:09:53 -0800 (PST) From: Peter Xu To: qemu-devel@nongnu.org Subject: [PATCH RFC 03/15] migration: Enable UFFD_FEATURE_THREAD_ID even without blocktime feat Date: Wed, 19 Jan 2022 16:09:17 +0800 Message-Id: <20220119080929.39485-4-peterx@redhat.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20220119080929.39485-1-peterx@redhat.com> References: <20220119080929.39485-1-peterx@redhat.com> MIME-Version: 1.0 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=peterx@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=peterx@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -34 X-Spam_score: -3.5 X-Spam_bar: --- X-Spam_report: (-3.5 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.7, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Juan Quintela , "Dr . David Alan Gilbert" , peterx@redhat.com, Leonardo Bras Soares Passos Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZM-MESSAGEID: 1642581182049100001 Content-Type: text/plain; charset="utf-8" This patch allows us to read the tid even without blocktime feature enabled. It's useful when tracing postcopy fault thread on faulted pages to show thr= ead id too with the address. Remove the comments - they're merely not helpful at all. Signed-off-by: Peter Xu Reviewed-by: Dr. David Alan Gilbert Reviewed-by: Juan Quintela --- migration/postcopy-ram.c | 14 ++++++-------- 1 file changed, 6 insertions(+), 8 deletions(-) diff --git a/migration/postcopy-ram.c b/migration/postcopy-ram.c index d18b5d05b2..2176ed68a5 100644 --- a/migration/postcopy-ram.c +++ b/migration/postcopy-ram.c @@ -283,15 +283,13 @@ static bool ufd_check_and_apply(int ufd, MigrationInc= omingState *mis) } =20 #ifdef UFFD_FEATURE_THREAD_ID - if (migrate_postcopy_blocktime() && mis && - UFFD_FEATURE_THREAD_ID & supported_features) { - /* kernel supports that feature */ - /* don't create blocktime_context if it exists */ - if (!mis->blocktime_ctx) { - mis->blocktime_ctx =3D blocktime_context_new(); - } - + if (UFFD_FEATURE_THREAD_ID & supported_features) { asked_features |=3D UFFD_FEATURE_THREAD_ID; + if (migrate_postcopy_blocktime()) { + if (!mis->blocktime_ctx) { + mis->blocktime_ctx =3D blocktime_context_new(); + } + } } #endif =20 --=20 2.32.0 From nobody Sat Apr 27 09:25:08 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1642581899814300.1390071461791; Wed, 19 Jan 2022 00:44:59 -0800 (PST) Received: from localhost ([::1]:59822 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nA6aO-0003IF-Vv for importer@patchew.org; Wed, 19 Jan 2022 03:44:57 -0500 Received: from eggs.gnu.org ([209.51.188.92]:58764) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nA62h-00071p-K8 for qemu-devel@nongnu.org; Wed, 19 Jan 2022 03:10:07 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:60939) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nA62e-0000ve-RD for qemu-devel@nongnu.org; Wed, 19 Jan 2022 03:10:07 -0500 Received: from mail-wm1-f69.google.com (mail-wm1-f69.google.com [209.85.128.69]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-592-7o66h_i1NKyRGreovZcf_w-1; Wed, 19 Jan 2022 03:09:59 -0500 Received: by mail-wm1-f69.google.com with SMTP id f187-20020a1c38c4000000b0034d5c66d8f5so1424803wma.5 for ; Wed, 19 Jan 2022 00:09:58 -0800 (PST) Received: from localhost.localdomain ([85.203.46.175]) by smtp.gmail.com with ESMTPSA id a20sm4351606wmb.27.2022.01.19.00.09.53 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Wed, 19 Jan 2022 00:09:56 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1642579800; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=MjqZSUin4KGk042s0HnlvgC6wrmCkM8/MM5iweJa6Cc=; b=E7FqOMyD7EyZUfkYdKBEHhQlNAOMSnU+lMymnL+Do0T5asoCY8b3qraJfhMeQSXXSyI/V1 Xt4bGOPMTl1B7hy6+4clu2HE0pGO50bzc463tcjKAuKxDXYW51Tg/3sqeEBeGyAEQFPQDS W6/pt0ddCLTN88TlNKPV0MVodR2voMM= X-MC-Unique: 7o66h_i1NKyRGreovZcf_w-1 X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=MjqZSUin4KGk042s0HnlvgC6wrmCkM8/MM5iweJa6Cc=; b=uVOxmkkCegoep/RGJfjajPU+S2YuyD7wIQ1I3kXIP2Jbnfa0g2kHMUAnqpzOqZeil0 jqG5vTogHEwUIbi/KueyO2Tl5CwrVmrNVwtFi1XH2c/ZR6NvZWdM2l8+Sy0tbcXREcpR PTsTWImkumfxd5vdEUVCRynem4awlck60d9FLYvOjSxUoSxObc1cGno37mcMEVUv9UAn EvgIvnPDQkWIIy7Q6pifkLWTx3BzK2tbJTxUv8YCzOeBa6A63v07zjUZkihqeeOnbqJo ZdR2ZsVj8WjdS09kz5p0ghK7mdFqcL1RicISRDz4VlPc8Mmq5zURVDja/6TEANon5Ke+ d0rg== X-Gm-Message-State: AOAM533Zxh1CEdjRyPwUPHAaJ9ZCHHokMsPt2A9CNA6PRFHeWJdRWEEs JrxLH/A4sMn7auZRr3Mv+sV1dPi27VytavSkhEacI918pQpWoq5SRloI3+FrGZgwky26XumcZqV KLrJ9zuf9MnYnRrofdEyYCpOETe/oSOw3PpMCMYaS5NFCoUwaupX6ZtG0kPLM5VE0 X-Received: by 2002:a5d:64c1:: with SMTP id f1mr15358217wri.633.1642579797489; Wed, 19 Jan 2022 00:09:57 -0800 (PST) X-Google-Smtp-Source: ABdhPJz+aZ0BIt09rnM/y9DndauFJob9gDwVcWyarrfIAAc0ofP98joFoAWbZaYkHz1MtlwX635+ow== X-Received: by 2002:a5d:64c1:: with SMTP id f1mr15358195wri.633.1642579797214; Wed, 19 Jan 2022 00:09:57 -0800 (PST) From: Peter Xu To: qemu-devel@nongnu.org Subject: [PATCH RFC 04/15] migration: Add postcopy_has_request() Date: Wed, 19 Jan 2022 16:09:18 +0800 Message-Id: <20220119080929.39485-5-peterx@redhat.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20220119080929.39485-1-peterx@redhat.com> References: <20220119080929.39485-1-peterx@redhat.com> MIME-Version: 1.0 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=peterx@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=peterx@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -28 X-Spam_score: -2.9 X-Spam_bar: -- X-Spam_report: (-2.9 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.7, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URG_BIZ=0.573 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Juan Quintela , "Dr . David Alan Gilbert" , peterx@redhat.com, Leonardo Bras Soares Passos Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZM-MESSAGEID: 1642581902349100001 Content-Type: text/plain; charset="utf-8" Add a helper to detect whether postcopy has pending request. Since at it, cleanup the code a bit, e.g. in unqueue_page() we shouldn't ne= ed to check it again on queue empty because we're the only one (besides cleanup code, which should never run during this process) that will take a request = off the list, so the request list can only grow but not shrink under the hood. Signed-off-by: Peter Xu Reviewed-by: Juan Quintela --- migration/ram.c | 45 ++++++++++++++++++++++++++++----------------- 1 file changed, 28 insertions(+), 17 deletions(-) diff --git a/migration/ram.c b/migration/ram.c index 94b0ad4234..dc6ba041fa 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -354,6 +354,12 @@ static RAMState *ram_state; =20 static NotifierWithReturnList precopy_notifier_list; =20 +/* Whether postcopy has queued requests? */ +static bool postcopy_has_request(RAMState *rs) +{ + return !QSIMPLEQ_EMPTY_ATOMIC(&rs->src_page_requests); +} + void precopy_infrastructure_init(void) { notifier_with_return_list_init(&precopy_notifier_list); @@ -1533,28 +1539,33 @@ static bool find_dirty_block(RAMState *rs, PageSear= chStatus *pss, bool *again) */ static RAMBlock *unqueue_page(RAMState *rs, ram_addr_t *offset) { + struct RAMSrcPageRequest *entry; RAMBlock *block =3D NULL; =20 - if (QSIMPLEQ_EMPTY_ATOMIC(&rs->src_page_requests)) { + if (!postcopy_has_request(rs)) { return NULL; } =20 QEMU_LOCK_GUARD(&rs->src_page_req_mutex); - if (!QSIMPLEQ_EMPTY(&rs->src_page_requests)) { - struct RAMSrcPageRequest *entry =3D - QSIMPLEQ_FIRST(&rs->src_page_requests); - block =3D entry->rb; - *offset =3D entry->offset; - - if (entry->len > TARGET_PAGE_SIZE) { - entry->len -=3D TARGET_PAGE_SIZE; - entry->offset +=3D TARGET_PAGE_SIZE; - } else { - memory_region_unref(block->mr); - QSIMPLEQ_REMOVE_HEAD(&rs->src_page_requests, next_req); - g_free(entry); - migration_consume_urgent_request(); - } + + /* + * This should _never_ change even after we take the lock, because no = one + * should be taking anything off the request list other than us. + */ + assert(postcopy_has_request(rs)); + + entry =3D QSIMPLEQ_FIRST(&rs->src_page_requests); + block =3D entry->rb; + *offset =3D entry->offset; + + if (entry->len > TARGET_PAGE_SIZE) { + entry->len -=3D TARGET_PAGE_SIZE; + entry->offset +=3D TARGET_PAGE_SIZE; + } else { + memory_region_unref(block->mr); + QSIMPLEQ_REMOVE_HEAD(&rs->src_page_requests, next_req); + g_free(entry); + migration_consume_urgent_request(); } =20 return block; @@ -2996,7 +3007,7 @@ static int ram_save_iterate(QEMUFile *f, void *opaque) t0 =3D qemu_clock_get_ns(QEMU_CLOCK_REALTIME); i =3D 0; while ((ret =3D qemu_file_rate_limit(f)) =3D=3D 0 || - !QSIMPLEQ_EMPTY(&rs->src_page_requests)) { + postcopy_has_request(rs)) { int pages; =20 if (qemu_file_get_error(f)) { --=20 2.32.0 From nobody Sat Apr 27 09:25:08 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1642582510984787.6909139104359; Wed, 19 Jan 2022 00:55:10 -0800 (PST) Received: from localhost ([::1]:40420 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nA6kH-0001a4-CB for importer@patchew.org; Wed, 19 Jan 2022 03:55:09 -0500 Received: from eggs.gnu.org ([209.51.188.92]:58788) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nA62i-00072W-3U for qemu-devel@nongnu.org; Wed, 19 Jan 2022 03:10:08 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]:24796) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nA62f-0000zt-R3 for qemu-devel@nongnu.org; Wed, 19 Jan 2022 03:10:07 -0500 Received: from mail-wm1-f70.google.com (mail-wm1-f70.google.com [209.85.128.70]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-247-1bKFoq8AO-mmyw_7Dh5MxA-1; Wed, 19 Jan 2022 03:10:03 -0500 Received: by mail-wm1-f70.google.com with SMTP id f7-20020a1cc907000000b0034b63f314ccso1328196wmb.6 for ; Wed, 19 Jan 2022 00:10:02 -0800 (PST) Received: from localhost.localdomain ([85.203.46.175]) by smtp.gmail.com with ESMTPSA id a20sm4351606wmb.27.2022.01.19.00.09.57 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Wed, 19 Jan 2022 00:10:00 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1642579804; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=ZgXSsm4TobC9VeROQOkPev+2EEsMYK5pH9blVW1j0Ww=; b=c4mmYXlj8eB/jv/+qPIXWOOz1TyY8eK60W2UcC4In/Gn2DNFXmZVcmo3QAiuJviBITcycB xKMEhhg5BmdRzvLZy/Xl+rmC6ZIM/HHBo9eoL1n++LZWd+jpLBL5qtmN/0yyxbf79ZmXmr qZDyt55rt4Rh2fH2od09bSRbjh5VPnM= X-MC-Unique: 1bKFoq8AO-mmyw_7Dh5MxA-1 X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=ZgXSsm4TobC9VeROQOkPev+2EEsMYK5pH9blVW1j0Ww=; b=jqjGAaoTS7SmSfXhxTRhc1lz+jR+aGT8T1f8NTmIIpWbZ/pvVDNRbQPZ+23hxgNjvP 4KF7wJpIPlqnHQie0Gc3LqWXxgngDeKZkgGL7N7ZReF7tZWxu/KkI2hFxl1m7CiGSqOW 5t5ur/Q8h6tWX7pjRcFJUcFQRafUmGZaoI0afGtOroNbr0ExNf62UOfb00r+fTNuAN1e xpMlraNNBIvMnl50LS+/cG+yxmX/V+qBV97qivan2nIppPaZUyOMmVffnjdHXYCZUQQz aN52Q7wkH0yrK6UFOH1aBA7/QcuJQrvQ9SgzsYiYgIWz9iyoMOgnpQzG+dDjXULpPqan FNdQ== X-Gm-Message-State: AOAM531ZeWHz1S24afYuL5lpvBfigFBJnXKDLBkYBgMNE81LOsu9kt6y tCocpjk1oIKY7B3OYQljxtYcJsEZJwG1/62NCJbz2O6FaXGn++eK8Iuv5DAHVONpys1VDiuqNBA cRiU3ANXA/t0B09aABIWVIyir4uIv1cqsWID8/Ubil7Jug5idmZuENec6yubeReis X-Received: by 2002:a5d:4ad2:: with SMTP id y18mr27753902wrs.141.1642579801528; Wed, 19 Jan 2022 00:10:01 -0800 (PST) X-Google-Smtp-Source: ABdhPJz9HNYyGr7BYHOnFdIxA2U9phmfkzn7k9/TNtuzKUXOKkGo/c9ZyY6ZR0yQMzSpP2DjeCDm3w== X-Received: by 2002:a5d:4ad2:: with SMTP id y18mr27753882wrs.141.1642579801277; Wed, 19 Jan 2022 00:10:01 -0800 (PST) From: Peter Xu To: qemu-devel@nongnu.org Subject: [PATCH RFC 05/15] migration: Simplify unqueue_page() Date: Wed, 19 Jan 2022 16:09:19 +0800 Message-Id: <20220119080929.39485-6-peterx@redhat.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20220119080929.39485-1-peterx@redhat.com> References: <20220119080929.39485-1-peterx@redhat.com> MIME-Version: 1.0 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=peterx@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=peterx@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -34 X-Spam_score: -3.5 X-Spam_bar: --- X-Spam_report: (-3.5 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.7, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Juan Quintela , "Dr . David Alan Gilbert" , peterx@redhat.com, Leonardo Bras Soares Passos Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZM-MESSAGEID: 1642582512870100001 Content-Type: text/plain; charset="utf-8" This patch simplifies unqueue_page() on both sides of it (itself, and calle= r). Firstly, due to the fact that right after unqueue_page() returned true, we'= ll definitely send a huge page (see ram_save_huge_page() call - it will _never_ exit before finish sending that huge page), so unqueue_page() does not need= to jump in small page size if huge page is enabled on the ramblock. IOW, it's destined that only the 1st 4K page will be valid, when unqueue the 2nd+ time we'll notice the whole huge page has already been sent anyway. Switching to operating on huge page reduces a lot of the loops of redundant unqueue_page= (). Meanwhile, drop the dirty check. It's not helpful to call test_bit() every time to jump over clean pages, as ram_save_host_page() has already done so, while in a faster way (see commit ba1b7c812c ("migration/ram: Optimize ram_save_host_page()", 2021-05-13)). So that's not necessary too. Drop the two tracepoints along the way - based on above analysis it's very possible that no one is really using it.. Signed-off-by: Peter Xu Reviewed-by: Dr. David Alan Gilbert Reviewed-by: Juan Quintela --- migration/ram.c | 34 ++++++++-------------------------- migration/trace-events | 2 -- 2 files changed, 8 insertions(+), 28 deletions(-) diff --git a/migration/ram.c b/migration/ram.c index dc6ba041fa..0df15ff663 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -1541,6 +1541,7 @@ static RAMBlock *unqueue_page(RAMState *rs, ram_addr_= t *offset) { struct RAMSrcPageRequest *entry; RAMBlock *block =3D NULL; + size_t page_size; =20 if (!postcopy_has_request(rs)) { return NULL; @@ -1557,10 +1558,13 @@ static RAMBlock *unqueue_page(RAMState *rs, ram_add= r_t *offset) entry =3D QSIMPLEQ_FIRST(&rs->src_page_requests); block =3D entry->rb; *offset =3D entry->offset; + page_size =3D qemu_ram_pagesize(block); + /* Each page request should only be multiple page size of the ramblock= */ + assert((entry->len % page_size) =3D=3D 0); =20 - if (entry->len > TARGET_PAGE_SIZE) { - entry->len -=3D TARGET_PAGE_SIZE; - entry->offset +=3D TARGET_PAGE_SIZE; + if (entry->len > page_size) { + entry->len -=3D page_size; + entry->offset +=3D page_size; } else { memory_region_unref(block->mr); QSIMPLEQ_REMOVE_HEAD(&rs->src_page_requests, next_req); @@ -1942,30 +1946,8 @@ static bool get_queued_page(RAMState *rs, PageSearch= Status *pss) { RAMBlock *block; ram_addr_t offset; - bool dirty; =20 - do { - block =3D unqueue_page(rs, &offset); - /* - * We're sending this page, and since it's postcopy nothing else - * will dirty it, and we must make sure it doesn't get sent again - * even if this queue request was received after the background - * search already sent it. - */ - if (block) { - unsigned long page; - - page =3D offset >> TARGET_PAGE_BITS; - dirty =3D test_bit(page, block->bmap); - if (!dirty) { - trace_get_queued_page_not_dirty(block->idstr, (uint64_t)of= fset, - page); - } else { - trace_get_queued_page(block->idstr, (uint64_t)offset, page= ); - } - } - - } while (block && !dirty); + block =3D unqueue_page(rs, &offset); =20 if (!block) { /* diff --git a/migration/trace-events b/migration/trace-events index e165687af2..3a9b3567ae 100644 --- a/migration/trace-events +++ b/migration/trace-events @@ -85,8 +85,6 @@ put_qlist_end(const char *field_name, const char *vmsd_na= me) "%s(%s)" qemu_file_fclose(void) "" =20 # ram.c -get_queued_page(const char *block_name, uint64_t tmp_offset, unsigned long= page_abs) "%s/0x%" PRIx64 " page_abs=3D0x%lx" -get_queued_page_not_dirty(const char *block_name, uint64_t tmp_offset, uns= igned long page_abs) "%s/0x%" PRIx64 " page_abs=3D0x%lx" migration_bitmap_sync_start(void) "" migration_bitmap_sync_end(uint64_t dirty_pages) "dirty_pages %" PRIu64 migration_bitmap_clear_dirty(char *str, uint64_t start, uint64_t size, uns= igned long page) "rb %s start 0x%"PRIx64" size 0x%"PRIx64" page 0x%lx" --=20 2.32.0 From nobody Sat Apr 27 09:25:08 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 164258036274895.90021266037263; Wed, 19 Jan 2022 00:19:22 -0800 (PST) Received: from localhost ([::1]:34700 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nA6Bd-0001dy-Bv for importer@patchew.org; Wed, 19 Jan 2022 03:19:21 -0500 Received: from eggs.gnu.org ([209.51.188.92]:58828) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nA62l-00076G-6t for qemu-devel@nongnu.org; Wed, 19 Jan 2022 03:10:11 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:51345) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nA62j-0001BF-2l for qemu-devel@nongnu.org; Wed, 19 Jan 2022 03:10:10 -0500 Received: from mail-wm1-f72.google.com (mail-wm1-f72.google.com [209.85.128.72]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-630-LjB4fwAmMzSMYfr28BXaUQ-1; Wed, 19 Jan 2022 03:10:07 -0500 Received: by mail-wm1-f72.google.com with SMTP id f187-20020a1c38c4000000b0034d5c66d8f5so1425079wma.5 for ; Wed, 19 Jan 2022 00:10:07 -0800 (PST) Received: from localhost.localdomain ([85.203.46.175]) by smtp.gmail.com with ESMTPSA id a20sm4351606wmb.27.2022.01.19.00.10.01 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Wed, 19 Jan 2022 00:10:04 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1642579808; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=o3RSjOQBlPTxLvQkAqgjPzeC1kjMjyIdJ01cEvje6X0=; b=DhOLD8e/MmeV+oB7fi9IkXarSElYvAKOlcmLgeR89I7TRWXKs0hJ5I1zR89DxAGD8uTInM DLgGUYaCqyUIZlAkbqdlMM35/TsKC3Dp2fN+fZNJAiQFdZMH9txXNWNObZj42dHINC0Pg6 +UcqPzn8sY901KO44hApejcOq/3dJGQ= X-MC-Unique: LjB4fwAmMzSMYfr28BXaUQ-1 X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=o3RSjOQBlPTxLvQkAqgjPzeC1kjMjyIdJ01cEvje6X0=; b=Ifj+SpMLGzoaRTd0aPB4d/iVXfMJ+b3BM8D2i4ApcEsvzU0k8acBmsj3ngwcjZYH1b fAV+pLhsuHlwyQ7gyNiiMLFX4e+Yai4+qqXojRUGYKFJZQyNRl3cZiib1OQN8k5XApb4 56fzL5uueNR/6ZhNzE9Sb6C7zL8JgDLLAZMD8LRm2I2pkhp8XoGFu2qEL3rKfgRxO+C9 qRrLld01hXWVeYMy5EX7rLBYhDBkaqohjLJvf36XYtaJioRX1kwrVhh7xyzGtWU8kmQE Q44y6utmfslLKL5e7aGHOTYn20FzQW3pvULp+xgW1h8WG8Zw3/zc+cuirFzCoW9CYFXy exAg== X-Gm-Message-State: AOAM532GPYjKdJxPNxkba/FFffMKdFBGqwuXkQb6F8si8oc/Amol5/Pc 2h6OKlmK4aQhyxZdSZ6YveASYtXDMprustTj2FFRHvcBOqIH9+GA/dVZ0ulhJsmarIRvgWNik60 Vvme/WCc2z04oyYeTvxRMsnT5xz+V+DQVouVeNAdvwDTSkryumRMl8QLsXkVgR7Ig X-Received: by 2002:a5d:5143:: with SMTP id u3mr3254555wrt.714.1642579805685; Wed, 19 Jan 2022 00:10:05 -0800 (PST) X-Google-Smtp-Source: ABdhPJx0BWv1+BjcDLjXD2pV4No6wG4J/SDLyeHxIIOsf3BwzAzgUkleJdktSw/iqo6sbM3Ul3aawQ== X-Received: by 2002:a5d:5143:: with SMTP id u3mr3254519wrt.714.1642579805349; Wed, 19 Jan 2022 00:10:05 -0800 (PST) From: Peter Xu To: qemu-devel@nongnu.org Subject: [PATCH RFC 06/15] migration: Move temp page setup and cleanup into separate functions Date: Wed, 19 Jan 2022 16:09:20 +0800 Message-Id: <20220119080929.39485-7-peterx@redhat.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20220119080929.39485-1-peterx@redhat.com> References: <20220119080929.39485-1-peterx@redhat.com> MIME-Version: 1.0 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=peterx@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=peterx@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -34 X-Spam_score: -3.5 X-Spam_bar: --- X-Spam_report: (-3.5 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.7, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Juan Quintela , "Dr . David Alan Gilbert" , peterx@redhat.com, Leonardo Bras Soares Passos Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZM-MESSAGEID: 1642580364108100001 Content-Type: text/plain; charset="utf-8" Temp pages will need to grow if we want to have multiple channels for postc= opy, because each channel will need its own temp page to cache huge page data. Before doing that, cleanup the related code. No functional change intended. Since at it, touch up the errno handling a little bit on the setup side. Signed-off-by: Peter Xu Reviewed-by: Dr. David Alan Gilbert Reviewed-by: Juan Quintela --- migration/postcopy-ram.c | 82 +++++++++++++++++++++++++--------------- 1 file changed, 51 insertions(+), 31 deletions(-) diff --git a/migration/postcopy-ram.c b/migration/postcopy-ram.c index 2176ed68a5..e662dd05cc 100644 --- a/migration/postcopy-ram.c +++ b/migration/postcopy-ram.c @@ -523,6 +523,19 @@ int postcopy_ram_incoming_init(MigrationIncomingState = *mis) return 0; } =20 +static void postcopy_temp_pages_cleanup(MigrationIncomingState *mis) +{ + if (mis->postcopy_tmp_page) { + munmap(mis->postcopy_tmp_page, mis->largest_page_size); + mis->postcopy_tmp_page =3D NULL; + } + + if (mis->postcopy_tmp_zero_page) { + munmap(mis->postcopy_tmp_zero_page, mis->largest_page_size); + mis->postcopy_tmp_zero_page =3D NULL; + } +} + /* * At the end of a migration where postcopy_ram_incoming_init was called. */ @@ -564,14 +577,8 @@ int postcopy_ram_incoming_cleanup(MigrationIncomingSta= te *mis) } } =20 - if (mis->postcopy_tmp_page) { - munmap(mis->postcopy_tmp_page, mis->largest_page_size); - mis->postcopy_tmp_page =3D NULL; - } - if (mis->postcopy_tmp_zero_page) { - munmap(mis->postcopy_tmp_zero_page, mis->largest_page_size); - mis->postcopy_tmp_zero_page =3D NULL; - } + postcopy_temp_pages_cleanup(mis); + trace_postcopy_ram_incoming_cleanup_blocktime( get_postcopy_total_blocktime()); =20 @@ -1082,6 +1089,40 @@ retry: return NULL; } =20 +static int postcopy_temp_pages_setup(MigrationIncomingState *mis) +{ + int err; + + mis->postcopy_tmp_page =3D mmap(NULL, mis->largest_page_size, + PROT_READ | PROT_WRITE, + MAP_PRIVATE | MAP_ANONYMOUS, -1, 0); + if (mis->postcopy_tmp_page =3D=3D MAP_FAILED) { + err =3D errno; + mis->postcopy_tmp_page =3D NULL; + error_report("%s: Failed to map postcopy_tmp_page %s", + __func__, strerror(err)); + return -err; + } + + /* + * Map large zero page when kernel can't use UFFDIO_ZEROPAGE for hugep= ages + */ + mis->postcopy_tmp_zero_page =3D mmap(NULL, mis->largest_page_size, + PROT_READ | PROT_WRITE, + MAP_PRIVATE | MAP_ANONYMOUS, -1, 0); + if (mis->postcopy_tmp_zero_page =3D=3D MAP_FAILED) { + err =3D errno; + mis->postcopy_tmp_zero_page =3D NULL; + error_report("%s: Failed to map large zero page %s", + __func__, strerror(err)); + return -err; + } + + memset(mis->postcopy_tmp_zero_page, '\0', mis->largest_page_size); + + return 0; +} + int postcopy_ram_incoming_setup(MigrationIncomingState *mis) { /* Open the fd for the kernel to give us userfaults */ @@ -1122,32 +1163,11 @@ int postcopy_ram_incoming_setup(MigrationIncomingSt= ate *mis) return -1; } =20 - mis->postcopy_tmp_page =3D mmap(NULL, mis->largest_page_size, - PROT_READ | PROT_WRITE, MAP_PRIVATE | - MAP_ANONYMOUS, -1, 0); - if (mis->postcopy_tmp_page =3D=3D MAP_FAILED) { - mis->postcopy_tmp_page =3D NULL; - error_report("%s: Failed to map postcopy_tmp_page %s", - __func__, strerror(errno)); + if (postcopy_temp_pages_setup(mis)) { + /* Error dumped in the sub-function */ return -1; } =20 - /* - * Map large zero page when kernel can't use UFFDIO_ZEROPAGE for hugep= ages - */ - mis->postcopy_tmp_zero_page =3D mmap(NULL, mis->largest_page_size, - PROT_READ | PROT_WRITE, - MAP_PRIVATE | MAP_ANONYMOUS, - -1, 0); - if (mis->postcopy_tmp_zero_page =3D=3D MAP_FAILED) { - int e =3D errno; - mis->postcopy_tmp_zero_page =3D NULL; - error_report("%s: Failed to map large zero page %s", - __func__, strerror(e)); - return -e; - } - memset(mis->postcopy_tmp_zero_page, '\0', mis->largest_page_size); - trace_postcopy_ram_enable_notify(); =20 return 0; --=20 2.32.0 From nobody Sat Apr 27 09:25:08 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1642583071522801.6094128820919; Wed, 19 Jan 2022 01:04:31 -0800 (PST) Received: from localhost ([::1]:49250 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nA6tJ-0007zO-VS for importer@patchew.org; Wed, 19 Jan 2022 04:04:30 -0500 Received: from eggs.gnu.org ([209.51.188.92]:58860) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nA62p-00079c-P4 for qemu-devel@nongnu.org; Wed, 19 Jan 2022 03:10:15 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:53558) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nA62n-0001DV-4h for qemu-devel@nongnu.org; Wed, 19 Jan 2022 03:10:15 -0500 Received: from mail-wm1-f72.google.com (mail-wm1-f72.google.com [209.85.128.72]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-660-LjFeZIh6OQyEw-2xdSaD6A-1; Wed, 19 Jan 2022 03:10:11 -0500 Received: by mail-wm1-f72.google.com with SMTP id 14-20020a05600c024e00b0034a83f7391aso1334874wmj.4 for ; Wed, 19 Jan 2022 00:10:11 -0800 (PST) Received: from localhost.localdomain ([85.203.46.175]) by smtp.gmail.com with ESMTPSA id a20sm4351606wmb.27.2022.01.19.00.10.05 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Wed, 19 Jan 2022 00:10:08 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1642579812; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=910DGuO+DFhT+8+Rx4es/kwyLJNyQzedlBlFYcKaZVk=; b=XAcUf7LDXuqitgRUnRb6NHdt9HNN5IOwN5NSAd7s6WLl0vMegKZOSz+nwBmGPhVXD9Wjrt agGH3zaYM4bHqjfbg2mYJcuKx324ehWQ91/wOBAp9BHl0MPTQCBBnJJ8wEo31JimhVQTBg BkBVCpx76EJHsKm4WB7/L+M1du5ldnA= X-MC-Unique: LjFeZIh6OQyEw-2xdSaD6A-1 X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=910DGuO+DFhT+8+Rx4es/kwyLJNyQzedlBlFYcKaZVk=; b=XZo8cG7f3hAVbn0ih5Hz/ZMkzeFIbgBKR/0O0MPJFlQqut58pxgfi6H7CLYkth7MD/ 11pcTCGRFMXgjz2tRD5VGOAOoR4s8UFD9PN20OMykZSAD17y012iiH7toqNJxXQE33IS Dc/S6QNDdCzs0jKA/N2ncAkEBWpVd+WWyXwB7F0pHFI5rLJp0rqqAX51R+ordNoFdn6o yMmoOazr8HXARNAJk+Tt8DUJCy3wzYC3tZRiEfsFFIH8GooOpX81vmthDTOjm3C03Pib LJcnnag90Zs3g94+l1IyNzUC/oKdOLSS1QHXQUQXmsPxcFU7vfaBHUtyGgXQ18NSh4vk YcQw== X-Gm-Message-State: AOAM530RODWcC3558Kjea0upKXAhdh1UaKSQGDyAhSuUlvAfCAJlHe8d guN5wm0h6Np1+ZkKc6VTeycCrO4xkSvbt/Eongzkje68SZQuqORQdNu+99kCZCbc22UqHMwKGrl Gjnm1gNMv9mn20xsUso1ooACezd251RA+6yf5D/YkScr+3dXmYV8pbyUQJHiTu7B2 X-Received: by 2002:a05:6000:1861:: with SMTP id d1mr12489515wri.145.1642579809709; Wed, 19 Jan 2022 00:10:09 -0800 (PST) X-Google-Smtp-Source: ABdhPJxfXoZG6rcq3dd1z3ny13D2HwVwAeIBU5Z508iC2aB3u71EHCj9ALfjcDx/bZ6Zy6EIiyG31g== X-Received: by 2002:a05:6000:1861:: with SMTP id d1mr12489487wri.145.1642579809310; Wed, 19 Jan 2022 00:10:09 -0800 (PST) From: Peter Xu To: qemu-devel@nongnu.org Subject: [PATCH RFC 07/15] migration: Introduce postcopy channels on dest node Date: Wed, 19 Jan 2022 16:09:21 +0800 Message-Id: <20220119080929.39485-8-peterx@redhat.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20220119080929.39485-1-peterx@redhat.com> References: <20220119080929.39485-1-peterx@redhat.com> MIME-Version: 1.0 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=peterx@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=peterx@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -34 X-Spam_score: -3.5 X-Spam_bar: --- X-Spam_report: (-3.5 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.7, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Juan Quintela , "Dr . David Alan Gilbert" , peterx@redhat.com, Leonardo Bras Soares Passos Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZM-MESSAGEID: 1642583073334100001 Content-Type: text/plain; charset="utf-8" Postcopy handles huge pages in a special way that currently we can only have one "channel" to transfer the page. It's because when we install pages using UFFDIO_COPY, we need to have the w= hole huge page ready, it also means we need to have a temp huge page when trying= to receive the whole content of the page. Currently all maintainance around this tmp page is global: firstly we'll allocate a temp huge page, then we maintain its status mostly within ram_load_postcopy(). To enable multiple channels for postcopy, the first thing we need to do is = to prepare N temp huge pages as caching, one for each channel. Meanwhile we need to maintain the tmp huge page status per-channel too. To give some example, some local variables maintained in ram_load_postcopy() are listed; they are responsible for maintaining temp huge page status: - all_zero: this keeps whether this huge page contains all zeros - target_pages: this counts how many target pages have been copied - host_page: this keeps the host ptr for the page to install Move all these fields to be together with the temp huge pages to form a new structure called PostcopyTmpPage. Then for each (future) postcopy channel,= we need one structure to keep the state around. For vanilla postcopy, obviously there's only one channel. It contains both precopy and postcopy pages. This patch teaches the dest migration node to start realize the possible nu= mber of postcopy channels by introducing the "postcopy_channels" variable. Its value is calculated when setup postcopy on dest node (during POSTCOPY_LISTEN phase). Vanilla postcopy will have channels=3D1, but when postcopy-preempt capabili= ty is enabled (in the future), we will boost it to 2 because even during partial sending of a precopy huge page we still want to preempt it and start sending the postcopy requested page right away (so we start to keep two temp huge pages; more if we want to enable multifd). In this patch there's a TODO ma= rked for that; so far the channels is always set to 1. We need to send one "host huge page" on one channel only and we cannot split them, because otherwise the data upon the same huge page can locate on more than one channel so we need more complicated logic to manage. One temp host huge page for each channel will be enough for us for now. Postcopy will still always use the index=3D0 huge page even after this patc= h. However it prepares for the latter patches where it can start to use multip= le channels (which needs src intervention, because only src knows which channe= l we should use). Signed-off-by: Peter Xu --- migration/migration.h | 35 +++++++++++++++++++++++++++- migration/postcopy-ram.c | 50 +++++++++++++++++++++++++++++----------- migration/ram.c | 43 +++++++++++++++++----------------- 3 files changed, 91 insertions(+), 37 deletions(-) diff --git a/migration/migration.h b/migration/migration.h index 8130b703eb..8bb2931312 100644 --- a/migration/migration.h +++ b/migration/migration.h @@ -45,6 +45,24 @@ struct PostcopyBlocktimeContext; */ #define CLEAR_BITMAP_SHIFT_MAX 31 =20 +/* This is an abstraction of a "temp huge page" for postcopy's purpose */ +typedef struct { + /* + * This points to a temporary huge page as a buffer for UFFDIO_COPY. = It's + * mmap()ed and needs to be freed when cleanup. + */ + void *tmp_huge_page; + /* + * This points to the host page we're going to install for this temp p= age. + * It tells us after we've received the whole page, where we should pu= t it. + */ + void *host_addr; + /* Number of small pages copied (in size of TARGET_PAGE_SIZE) */ + int target_pages; + /* Whether this page contains all zeros */ + bool all_zero; +} PostcopyTmpPage; + /* State for the incoming migration */ struct MigrationIncomingState { QEMUFile *from_src_file; @@ -81,7 +99,22 @@ struct MigrationIncomingState { QemuMutex rp_mutex; /* We send replies from multiple threads */ /* RAMBlock of last request sent to source */ RAMBlock *last_rb; - void *postcopy_tmp_page; + /* + * Number of postcopy channels including the default precopy channel, = so + * vanilla postcopy will only contain one channel which contain both + * precopy and postcopy streams. + * + * This is calculated when the src requests to enable postcopy but bef= ore + * it starts. Its value can depend on e.g. whether postcopy preemptio= n is + * enabled. + */ + int postcopy_channels; + /* + * An array of temp host huge pages to be used, one for each postcopy + * channel. + */ + PostcopyTmpPage *postcopy_tmp_pages; + /* This is shared for all postcopy channels */ void *postcopy_tmp_zero_page; /* PostCopyFD's for external userfaultfds & handlers of shared memory = */ GArray *postcopy_remote_fds; diff --git a/migration/postcopy-ram.c b/migration/postcopy-ram.c index e662dd05cc..d78e1b9373 100644 --- a/migration/postcopy-ram.c +++ b/migration/postcopy-ram.c @@ -525,9 +525,18 @@ int postcopy_ram_incoming_init(MigrationIncomingState = *mis) =20 static void postcopy_temp_pages_cleanup(MigrationIncomingState *mis) { - if (mis->postcopy_tmp_page) { - munmap(mis->postcopy_tmp_page, mis->largest_page_size); - mis->postcopy_tmp_page =3D NULL; + int i; + + if (mis->postcopy_tmp_pages) { + for (i =3D 0; i < mis->postcopy_channels; i++) { + if (mis->postcopy_tmp_pages[i].tmp_huge_page) { + munmap(mis->postcopy_tmp_pages[i].tmp_huge_page, + mis->largest_page_size); + mis->postcopy_tmp_pages[i].tmp_huge_page =3D NULL; + } + } + g_free(mis->postcopy_tmp_pages); + mis->postcopy_tmp_pages =3D NULL; } =20 if (mis->postcopy_tmp_zero_page) { @@ -1091,17 +1100,30 @@ retry: =20 static int postcopy_temp_pages_setup(MigrationIncomingState *mis) { - int err; - - mis->postcopy_tmp_page =3D mmap(NULL, mis->largest_page_size, - PROT_READ | PROT_WRITE, - MAP_PRIVATE | MAP_ANONYMOUS, -1, 0); - if (mis->postcopy_tmp_page =3D=3D MAP_FAILED) { - err =3D errno; - mis->postcopy_tmp_page =3D NULL; - error_report("%s: Failed to map postcopy_tmp_page %s", - __func__, strerror(err)); - return -err; + PostcopyTmpPage *tmp_page; + int err, i, channels; + void *temp_page; + + /* TODO: will be boosted when enable postcopy preemption */ + mis->postcopy_channels =3D 1; + + channels =3D mis->postcopy_channels; + mis->postcopy_tmp_pages =3D g_malloc0(sizeof(PostcopyTmpPage) * channe= ls); + + for (i =3D 0; i < channels; i++) { + tmp_page =3D &mis->postcopy_tmp_pages[i]; + temp_page =3D mmap(NULL, mis->largest_page_size, PROT_READ | PROT_= WRITE, + MAP_PRIVATE | MAP_ANONYMOUS, -1, 0); + if (temp_page =3D=3D MAP_FAILED) { + err =3D errno; + error_report("%s: Failed to map postcopy_tmp_pages[%d]: %s", + __func__, i, strerror(err)); + return -err; + } + tmp_page->tmp_huge_page =3D temp_page; + /* Initialize default states for each tmp page */ + tmp_page->all_zero =3D true; + tmp_page->target_pages =3D 0; } =20 /* diff --git a/migration/ram.c b/migration/ram.c index 0df15ff663..930e722e39 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -3639,11 +3639,8 @@ static int ram_load_postcopy(QEMUFile *f) bool place_needed =3D false; bool matches_target_page_size =3D false; MigrationIncomingState *mis =3D migration_incoming_get_current(); - /* Temporary page that is later 'placed' */ - void *postcopy_host_page =3D mis->postcopy_tmp_page; - void *host_page =3D NULL; - bool all_zero =3D true; - int target_pages =3D 0; + /* Currently we only use channel 0. TODO: use all the channels */ + PostcopyTmpPage *tmp_page =3D &mis->postcopy_tmp_pages[0]; =20 while (!ret && !(flags & RAM_SAVE_FLAG_EOS)) { ram_addr_t addr; @@ -3687,7 +3684,7 @@ static int ram_load_postcopy(QEMUFile *f) ret =3D -EINVAL; break; } - target_pages++; + tmp_page->target_pages++; matches_target_page_size =3D block->page_size =3D=3D TARGET_PA= GE_SIZE; /* * Postcopy requires that we place whole host pages atomically; @@ -3699,15 +3696,16 @@ static int ram_load_postcopy(QEMUFile *f) * however the source ensures it always sends all the componen= ts * of a host page in one chunk. */ - page_buffer =3D postcopy_host_page + + page_buffer =3D tmp_page->tmp_huge_page + host_page_offset_from_ram_block_offset(block, ad= dr); /* If all TP are zero then we can optimise the place */ - if (target_pages =3D=3D 1) { - host_page =3D host_page_from_ram_block_offset(block, addr); - } else if (host_page !=3D host_page_from_ram_block_offset(bloc= k, - addr))= { + if (tmp_page->target_pages =3D=3D 1) { + tmp_page->host_addr =3D + host_page_from_ram_block_offset(block, addr); + } else if (tmp_page->host_addr !=3D + host_page_from_ram_block_offset(block, addr)) { /* not the 1st TP within the HP */ - error_report("Non-same host page %p/%p", host_page, + error_report("Non-same host page %p/%p", tmp_page->host_ad= dr, host_page_from_ram_block_offset(block, addr)); ret =3D -EINVAL; break; @@ -3717,10 +3715,11 @@ static int ram_load_postcopy(QEMUFile *f) * If it's the last part of a host page then we place the host * page */ - if (target_pages =3D=3D (block->page_size / TARGET_PAGE_SIZE))= { + if (tmp_page->target_pages =3D=3D + (block->page_size / TARGET_PAGE_SIZE)) { place_needed =3D true; } - place_source =3D postcopy_host_page; + place_source =3D tmp_page->tmp_huge_page; } =20 switch (flags & ~RAM_SAVE_FLAG_CONTINUE) { @@ -3734,12 +3733,12 @@ static int ram_load_postcopy(QEMUFile *f) memset(page_buffer, ch, TARGET_PAGE_SIZE); } if (ch) { - all_zero =3D false; + tmp_page->all_zero =3D false; } break; =20 case RAM_SAVE_FLAG_PAGE: - all_zero =3D false; + tmp_page->all_zero =3D false; if (!matches_target_page_size) { /* For huge pages, we always use temporary buffer */ qemu_get_buffer(f, page_buffer, TARGET_PAGE_SIZE); @@ -3757,7 +3756,7 @@ static int ram_load_postcopy(QEMUFile *f) } break; case RAM_SAVE_FLAG_COMPRESS_PAGE: - all_zero =3D false; + tmp_page->all_zero =3D false; len =3D qemu_get_be32(f); if (len < 0 || len > compressBound(TARGET_PAGE_SIZE)) { error_report("Invalid compressed data length: %d", len); @@ -3789,16 +3788,16 @@ static int ram_load_postcopy(QEMUFile *f) } =20 if (!ret && place_needed) { - if (all_zero) { - ret =3D postcopy_place_page_zero(mis, host_page, block); + if (tmp_page->all_zero) { + ret =3D postcopy_place_page_zero(mis, tmp_page->host_addr,= block); } else { - ret =3D postcopy_place_page(mis, host_page, place_source, + ret =3D postcopy_place_page(mis, tmp_page->host_addr, plac= e_source, block); } place_needed =3D false; - target_pages =3D 0; + tmp_page->target_pages =3D 0; /* Assume we have a zero page until we detect something differ= ent */ - all_zero =3D true; + tmp_page->all_zero =3D true; } } =20 --=20 2.32.0 From nobody Sat Apr 27 09:25:08 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1642580365395506.6505138224728; Wed, 19 Jan 2022 00:19:25 -0800 (PST) Received: from localhost ([::1]:34928 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nA6Bg-0001u2-68 for importer@patchew.org; Wed, 19 Jan 2022 03:19:24 -0500 Received: from eggs.gnu.org ([209.51.188.92]:58892) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nA62u-0007EF-1n for qemu-devel@nongnu.org; Wed, 19 Jan 2022 03:10:20 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]:25046) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nA62r-0001HU-Qj for qemu-devel@nongnu.org; Wed, 19 Jan 2022 03:10:19 -0500 Received: from mail-wm1-f72.google.com (mail-wm1-f72.google.com [209.85.128.72]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-657-BFwdMvdvODaSmfWEPCnqbg-1; Wed, 19 Jan 2022 03:10:15 -0500 Received: by mail-wm1-f72.google.com with SMTP id 20-20020a05600c22d400b00349067fe7b7so789275wmg.5 for ; Wed, 19 Jan 2022 00:10:15 -0800 (PST) Received: from localhost.localdomain ([85.203.46.175]) by smtp.gmail.com with ESMTPSA id a20sm4351606wmb.27.2022.01.19.00.10.09 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Wed, 19 Jan 2022 00:10:12 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1642579816; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=hEYlCjTDFsa1y8qf+67Mx05+32zK1b6BW22u1QcMTyk=; b=hhYlatucgfAoI64w9ys3kYxLlLM9riihYiUDFsk8C9+rZ6A8FLdYVIrsnsoLcvmrCgCEDh iqbLQ4lmIkZir71K68sUQH9qPxdqPvUvpWAOF9SaDBypRaUhjbtr5+xoTvMworew5AbRiV UdihzzN5q8qpV6uiCqo08t96oyWiIDI= X-MC-Unique: BFwdMvdvODaSmfWEPCnqbg-1 X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=hEYlCjTDFsa1y8qf+67Mx05+32zK1b6BW22u1QcMTyk=; b=WI2rVOwwDe/dogQkEhSp5ddvy+u4oTXct5vZXhAd778AAipIehEpdtBxznDH8a0Xjb dCxUdpAJ/fT0XE1SDi6vGpOlrIFc2Q1q4aVAwx2BVQq7Y1Ck8G7XgaNUjW2p0u1iAQ9Q n3BqY1P/pDx7T41Q+iochN+sCDrbjRUghCwRbvQ3ks0YwMgE2KsN5gWtcP1sMOG4XZdR AWcN3hr6eQgKwoihY6dVw439ILkr5vKwA/zibWFOhbvCufuVWX4p8LYzH91r9DI+OILS YiHS3tLZKGSCatPs/UAnKLy5goh26bA1pDu50NPJsb7hM0nA6fmxLmy+qfzbeTVSrdkg BlXA== X-Gm-Message-State: AOAM533KDBpVctp8IXm41pg61ktb3iCrpIFKfp1AqwNjIgiVzXiB4bHI y/k7qNx39W2ZMx3qcPOQ4slQQh71W76E5nb0kzj+MPNDDjVm+RCwmYVEkRV3Qj8X6zHWbUYeQP1 6pgE2vjuMO1pJ5+39eEH81FKfJdhQaQ4o9vLqQjkbwCfa5V6bTpYNzIGh6/IQq1bh X-Received: by 2002:a05:6000:144f:: with SMTP id v15mr6623040wrx.514.1642579813685; Wed, 19 Jan 2022 00:10:13 -0800 (PST) X-Google-Smtp-Source: ABdhPJyj/cj8r3cytrkCQE3KIdkiOXJMSkyls5Bad355NP6S3n63iEkbYFiuKHMYVJu8rvojOrm4Fg== X-Received: by 2002:a05:6000:144f:: with SMTP id v15mr6623013wrx.514.1642579813368; Wed, 19 Jan 2022 00:10:13 -0800 (PST) From: Peter Xu To: qemu-devel@nongnu.org Subject: [PATCH RFC 08/15] migration: Dump ramblock and offset too when non-same-page detected Date: Wed, 19 Jan 2022 16:09:22 +0800 Message-Id: <20220119080929.39485-9-peterx@redhat.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20220119080929.39485-1-peterx@redhat.com> References: <20220119080929.39485-1-peterx@redhat.com> MIME-Version: 1.0 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=peterx@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=peterx@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -34 X-Spam_score: -3.5 X-Spam_bar: --- X-Spam_report: (-3.5 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.7, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Juan Quintela , "Dr . David Alan Gilbert" , peterx@redhat.com, Leonardo Bras Soares Passos Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZM-MESSAGEID: 1642580366092100003 Content-Type: text/plain; charset="utf-8" In ram_load_postcopy() we'll try to detect non-same-page case and dump erro= r. This error is very helpful for debugging. Adding ramblock & offset into the error log too. Signed-off-by: Peter Xu Reviewed-by: Dr. David Alan Gilbert --- migration/ram.c | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-) diff --git a/migration/ram.c b/migration/ram.c index 930e722e39..3f823ffffc 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -3705,8 +3705,12 @@ static int ram_load_postcopy(QEMUFile *f) } else if (tmp_page->host_addr !=3D host_page_from_ram_block_offset(block, addr)) { /* not the 1st TP within the HP */ - error_report("Non-same host page %p/%p", tmp_page->host_ad= dr, - host_page_from_ram_block_offset(block, addr)); + error_report("Non-same host page detected. Target host pa= ge %p, " + "received host page %p " + "(rb %s offset 0x"RAM_ADDR_FMT" target_pages = %d)", + tmp_page->host_addr, + host_page_from_ram_block_offset(block, addr), + block->idstr, addr, tmp_page->target_pages); ret =3D -EINVAL; break; } --=20 2.32.0 From nobody Sat Apr 27 09:25:08 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1642580804581603.8659633818933; Wed, 19 Jan 2022 00:26:44 -0800 (PST) Received: from localhost ([::1]:43402 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nA6Il-0008C6-JK for importer@patchew.org; Wed, 19 Jan 2022 03:26:43 -0500 Received: from eggs.gnu.org ([209.51.188.92]:58942) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nA630-0007Ip-9z for qemu-devel@nongnu.org; Wed, 19 Jan 2022 03:10:26 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:60009) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nA62x-0001LP-Jh for qemu-devel@nongnu.org; Wed, 19 Jan 2022 03:10:25 -0500 Received: from mail-wm1-f71.google.com (mail-wm1-f71.google.com [209.85.128.71]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-672-Po6pliQiOQCVm3LsD9cNCg-1; Wed, 19 Jan 2022 03:10:19 -0500 Received: by mail-wm1-f71.google.com with SMTP id n14-20020a7bcbce000000b003488820f0d9so1322902wmi.8 for ; Wed, 19 Jan 2022 00:10:19 -0800 (PST) Received: from localhost.localdomain ([85.203.46.175]) by smtp.gmail.com with ESMTPSA id a20sm4351606wmb.27.2022.01.19.00.10.13 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Wed, 19 Jan 2022 00:10:17 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1642579823; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=ban/4LXzHGp/g8/SKxR9D6U/gtgCr2d4yvcIU42kzb0=; b=EAw7wSbxtQQnzsbp4DTF0VefBB63rqPqdu/wxE60M0dstZ4sxcWklAN225/YADn3OvD3zy 2uVIUTSGavk0Bu+Xhg3Y5QW5PCuVSI6sbUkH0275bwqbAwmnYZtREwyudoHXYTDwl/q7rO PEHojYoGA02ooliGo6HZU8nHhkgHqsw= X-MC-Unique: Po6pliQiOQCVm3LsD9cNCg-1 X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=ban/4LXzHGp/g8/SKxR9D6U/gtgCr2d4yvcIU42kzb0=; b=fNtqkMhy9Md8j4mhaJQN7ZxHXO39ejEB4U6vvjdOuWABGmPwXhZgxoZim4RN6OnPdY 0si0FuUNh4KgW9mp3gvlGksX23Ok3CZEyuX/PinQbyS/Wf9sHEOttKfoFPJhXy03aAGm Cco+/y2AHvsEnW42tiPlyJAQRhPLf7B3Xxa+JcdgYW8XG2/IO7Kd1+AIz9CNNzbkDcOv vX1I7q4mn7NiwQrrs82ftiPggxbsHOrRry+k9Pns4gl1+kX5zSFzh3bqFumtk6iN4uzC 4ZmgaUurrK0y7JrwdWYzUZkLyWrxA7rSilXNBiQO6uCcNgQlF7BH4JMPdaZjrYCQxZla cWvw== X-Gm-Message-State: AOAM533EiAylt/wjOgjXQHm9dOVcvQewqcOv/q5FAZl75gl2A45UjLhh 4jlJvpjVgD1oZTEfUGEd/3Pz3y93dFj0LaYL9PuPadlGAj4PT9dpBYY50jUsRfTeTEQ/KpFXHnU E0ktCuL9sCTG83HPhsb0ii5hI73fWHaRbrBU2hrR5PdE5qy4nE6GJ1gbJ53SRhcFq X-Received: by 2002:a05:6000:2ac:: with SMTP id l12mr17834452wry.165.1642579817808; Wed, 19 Jan 2022 00:10:17 -0800 (PST) X-Google-Smtp-Source: ABdhPJwHGakgjf7jZce7FiWR+S94mdyMdJvnPKzU2aYWozZKneioJl+JGy9gpoHE/ncATXRsLg22mg== X-Received: by 2002:a05:6000:2ac:: with SMTP id l12mr17834424wry.165.1642579817516; Wed, 19 Jan 2022 00:10:17 -0800 (PST) From: Peter Xu To: qemu-devel@nongnu.org Subject: [PATCH RFC 09/15] migration: Add postcopy_thread_create() Date: Wed, 19 Jan 2022 16:09:23 +0800 Message-Id: <20220119080929.39485-10-peterx@redhat.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20220119080929.39485-1-peterx@redhat.com> References: <20220119080929.39485-1-peterx@redhat.com> MIME-Version: 1.0 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=peterx@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=peterx@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -34 X-Spam_score: -3.5 X-Spam_bar: --- X-Spam_report: (-3.5 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.7, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Juan Quintela , "Dr . David Alan Gilbert" , peterx@redhat.com, Leonardo Bras Soares Passos Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZM-MESSAGEID: 1642580806749100003 Content-Type: text/plain; charset="utf-8" Postcopy create threads. A common manner is we init a sem and use it to sync with the thread. Namely, we have fault_thread_sem and listen_thread_sem and they're only used for this. Make it a shared infrastructure so it's easier to create yet another thread. Signed-off-by: Peter Xu Reviewed-by: Dr. David Alan Gilbert --- migration/migration.h | 5 ++--- migration/postcopy-ram.c | 19 +++++++++++++------ migration/postcopy-ram.h | 4 ++++ migration/savevm.c | 12 +++--------- 4 files changed, 22 insertions(+), 18 deletions(-) diff --git a/migration/migration.h b/migration/migration.h index 8bb2931312..35e7f7babe 100644 --- a/migration/migration.h +++ b/migration/migration.h @@ -70,7 +70,8 @@ struct MigrationIncomingState { /* A hook to allow cleanup at the end of incoming migration */ void *transport_data; void (*transport_cleanup)(void *data); - + /* Used to sync thread creations */ + QemuSemaphore thread_sync_sem; /* * Free at the start of the main state load, set as the main thread fi= nishes * loading state. @@ -83,13 +84,11 @@ struct MigrationIncomingState { size_t largest_page_size; bool have_fault_thread; QemuThread fault_thread; - QemuSemaphore fault_thread_sem; /* Set this when we want the fault thread to quit */ bool fault_thread_quit; =20 bool have_listen_thread; QemuThread listen_thread; - QemuSemaphore listen_thread_sem; =20 /* For the kernel to send us notifications */ int userfault_fd; diff --git a/migration/postcopy-ram.c b/migration/postcopy-ram.c index d78e1b9373..88c832eeba 100644 --- a/migration/postcopy-ram.c +++ b/migration/postcopy-ram.c @@ -77,6 +77,16 @@ int postcopy_notify(enum PostcopyNotifyReason reason, Er= ror **errp) &pnd); } =20 +void postcopy_thread_create(MigrationIncomingState *mis, + QemuThread *thread, const char *name, + void *(*fn)(void *), int joinable) +{ + qemu_sem_init(&mis->thread_sync_sem, 0); + qemu_thread_create(thread, name, fn, mis, joinable); + qemu_sem_wait(&mis->thread_sync_sem); + qemu_sem_destroy(&mis->thread_sync_sem); +} + /* Postcopy needs to detect accesses to pages that haven't yet been copied * across, and efficiently map new pages in, the techniques for doing this * are target OS specific. @@ -901,7 +911,7 @@ static void *postcopy_ram_fault_thread(void *opaque) trace_postcopy_ram_fault_thread_entry(); rcu_register_thread(); mis->last_rb =3D NULL; /* last RAMBlock we sent part of */ - qemu_sem_post(&mis->fault_thread_sem); + qemu_sem_post(&mis->thread_sync_sem); =20 struct pollfd *pfd; size_t pfd_len =3D 2 + mis->postcopy_remote_fds->len; @@ -1172,11 +1182,8 @@ int postcopy_ram_incoming_setup(MigrationIncomingSta= te *mis) return -1; } =20 - qemu_sem_init(&mis->fault_thread_sem, 0); - qemu_thread_create(&mis->fault_thread, "postcopy/fault", - postcopy_ram_fault_thread, mis, QEMU_THREAD_JOINABL= E); - qemu_sem_wait(&mis->fault_thread_sem); - qemu_sem_destroy(&mis->fault_thread_sem); + postcopy_thread_create(mis, &mis->fault_thread, "postcopy/fault", + postcopy_ram_fault_thread, QEMU_THREAD_JOINABLE= ); mis->have_fault_thread =3D true; =20 /* Mark so that we get notified of accesses to unwritten areas */ diff --git a/migration/postcopy-ram.h b/migration/postcopy-ram.h index 6d2b3cf124..07684c0e1d 100644 --- a/migration/postcopy-ram.h +++ b/migration/postcopy-ram.h @@ -135,6 +135,10 @@ void postcopy_remove_notifier(NotifierWithReturn *n); /* Call the notifier list set by postcopy_add_start_notifier */ int postcopy_notify(enum PostcopyNotifyReason reason, Error **errp); =20 +void postcopy_thread_create(MigrationIncomingState *mis, + QemuThread *thread, const char *name, + void *(*fn)(void *), int joinable); + struct PostCopyFD; =20 /* ufd is a pointer to the struct uffd_msg *TODO: more Portable! */ diff --git a/migration/savevm.c b/migration/savevm.c index 3b8f565b14..3342b74c24 100644 --- a/migration/savevm.c +++ b/migration/savevm.c @@ -1862,7 +1862,7 @@ static void *postcopy_ram_listen_thread(void *opaque) =20 migrate_set_state(&mis->state, MIGRATION_STATUS_ACTIVE, MIGRATION_STATUS_POSTCOPY_ACTIVE); - qemu_sem_post(&mis->listen_thread_sem); + qemu_sem_post(&mis->thread_sync_sem); trace_postcopy_ram_listen_thread_start(); =20 rcu_register_thread(); @@ -1987,14 +1987,8 @@ static int loadvm_postcopy_handle_listen(MigrationIn= comingState *mis) } =20 mis->have_listen_thread =3D true; - /* Start up the listening thread and wait for it to signal ready */ - qemu_sem_init(&mis->listen_thread_sem, 0); - qemu_thread_create(&mis->listen_thread, "postcopy/listen", - postcopy_ram_listen_thread, NULL, - QEMU_THREAD_DETACHED); - qemu_sem_wait(&mis->listen_thread_sem); - qemu_sem_destroy(&mis->listen_thread_sem); - + postcopy_thread_create(mis, &mis->listen_thread, "postcopy/listen", + postcopy_ram_listen_thread, QEMU_THREAD_DETACHE= D); trace_loadvm_postcopy_handle_listen("return"); =20 return 0; --=20 2.32.0 From nobody Sat Apr 27 09:25:08 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1642581273961358.0151945271007; Wed, 19 Jan 2022 00:34:33 -0800 (PST) Received: from localhost ([::1]:51724 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nA6QK-0005mP-DG for importer@patchew.org; Wed, 19 Jan 2022 03:34:32 -0500 Received: from eggs.gnu.org ([209.51.188.92]:58962) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nA631-0007KC-Mp for qemu-devel@nongnu.org; Wed, 19 Jan 2022 03:10:27 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]:46092) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nA62z-0001Lj-4k for qemu-devel@nongnu.org; Wed, 19 Jan 2022 03:10:27 -0500 Received: from mail-wm1-f69.google.com (mail-wm1-f69.google.com [209.85.128.69]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-623-a4UfuOjcOnWTMSuVe3YjaQ-1; Wed, 19 Jan 2022 03:10:23 -0500 Received: by mail-wm1-f69.google.com with SMTP id o3-20020a05600c4fc300b0034aee9534bdso801356wmq.2 for ; Wed, 19 Jan 2022 00:10:23 -0800 (PST) Received: from localhost.localdomain ([85.203.46.175]) by smtp.gmail.com with ESMTPSA id a20sm4351606wmb.27.2022.01.19.00.10.17 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Wed, 19 Jan 2022 00:10:21 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1642579824; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=OZVkrMSngFS5rGQULjw+Q25CAXqrhJkHW+x+P4pPtk0=; b=Fhh4oAeqJU5cvpgrz25zehVXOfVODu/ImiuOaMZTd1C5HQjDeJC2Re7H7pURizmvUo03yF cavyjQBt/BVXnQOv0Vijt5ZhABjVLo3VG7+OnqOA/w26xLNiQYJ1+yFbeEBBwXpr5Pip/X KaAfXNOwviAdy1JRp9JAaEfJVuNV118= X-MC-Unique: a4UfuOjcOnWTMSuVe3YjaQ-1 X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=OZVkrMSngFS5rGQULjw+Q25CAXqrhJkHW+x+P4pPtk0=; b=2UyUGZjRb1fhK2z6HhNk3u0+wTt1OOB08JKqzQqv/oEfssCRtbyqEXFclZ6RdPbcR3 3CLiPdwAH1P5stcMCkq8HBTUoekEsK1d66Q23y3DtP17ewSH8LkyLCLklsKYNfGCwcu2 GKysH4TJ2RIojmTO3KEbVmZOZcxcCxTm2VJR9I4dwnHMnrysPcx9t3Xnl06aLDPR5HLg BcjlBVluzqOeUgRJTIqtKPBAuKvnSH4Ljtg+HGDAULNQn9yH7uB/RDbXkNFIBB1CWGJf U1KMwBolASPwKA1B95cfEVZ2hBXorUTIjriDWfV/TVPN1h56cNX5Kxit820E1NYP9pfx dYQw== X-Gm-Message-State: AOAM531xTJjpTSHtM+zFJMDxG/p5LynM01p1Nw+g0jjZiPw/GSmvrFYl jowFdSuMq61CqnvP2kWO2+wLnO09RWHexT9OiEOvkI/OvCXE3W2MhqYlXNVa3f3dxMHN2GlD6wX 4ATsJtmj+k1Fq0W4RDJTCeojqeQSkIjWIlTd4+FKMygr/3X/9/A7pmAhf7Fh7f6Hr X-Received: by 2002:a5d:5253:: with SMTP id k19mr27827373wrc.228.1642579821921; Wed, 19 Jan 2022 00:10:21 -0800 (PST) X-Google-Smtp-Source: ABdhPJyMp8cvXIhMIQDd/KIuJlaDjnGay3q0d5NZ2Zt3AGtPRSE6At12L5wzeY/1arKdEVMVti8Z2w== X-Received: by 2002:a5d:5253:: with SMTP id k19mr27827356wrc.228.1642579821638; Wed, 19 Jan 2022 00:10:21 -0800 (PST) From: Peter Xu To: qemu-devel@nongnu.org Subject: [PATCH RFC 10/15] migration: Move static var in ram_block_from_stream() into global Date: Wed, 19 Jan 2022 16:09:24 +0800 Message-Id: <20220119080929.39485-11-peterx@redhat.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20220119080929.39485-1-peterx@redhat.com> References: <20220119080929.39485-1-peterx@redhat.com> MIME-Version: 1.0 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=peterx@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=peterx@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -34 X-Spam_score: -3.5 X-Spam_bar: --- X-Spam_report: (-3.5 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.7, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Juan Quintela , "Dr . David Alan Gilbert" , peterx@redhat.com, Leonardo Bras Soares Passos Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZM-MESSAGEID: 1642581274709100001 Content-Type: text/plain; charset="utf-8" Static variable is very unfriendly to threading of ram_block_from_stream(). Move it into MigrationIncomingState. Make the incoming state pointer to be passed over to ram_block_from_stream(= ) on both caller sites. Signed-off-by: Peter Xu Reviewed-by: Dr. David Alan Gilbert --- migration/migration.h | 3 ++- migration/ram.c | 13 +++++++++---- 2 files changed, 11 insertions(+), 5 deletions(-) diff --git a/migration/migration.h b/migration/migration.h index 35e7f7babe..34b79cb961 100644 --- a/migration/migration.h +++ b/migration/migration.h @@ -66,7 +66,8 @@ typedef struct { /* State for the incoming migration */ struct MigrationIncomingState { QEMUFile *from_src_file; - + /* Previously received RAM's RAMBlock pointer */ + RAMBlock *last_recv_block; /* A hook to allow cleanup at the end of incoming migration */ void *transport_data; void (*transport_cleanup)(void *data); diff --git a/migration/ram.c b/migration/ram.c index 3f823ffffc..3a7d943f9c 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -3183,12 +3183,14 @@ static int load_xbzrle(QEMUFile *f, ram_addr_t addr= , void *host) * * Returns a pointer from within the RCU-protected ram_list. * + * @mis: the migration incoming state pointer * @f: QEMUFile where to read the data from * @flags: Page flags (mostly to see if it's a continuation of previous bl= ock) */ -static inline RAMBlock *ram_block_from_stream(QEMUFile *f, int flags) +static inline RAMBlock *ram_block_from_stream(MigrationIncomingState *mis, + QEMUFile *f, int flags) { - static RAMBlock *block; + RAMBlock *block =3D mis->last_recv_block; char id[256]; uint8_t len; =20 @@ -3215,6 +3217,8 @@ static inline RAMBlock *ram_block_from_stream(QEMUFil= e *f, int flags) return NULL; } =20 + mis->last_recv_block =3D block; + return block; } =20 @@ -3667,7 +3671,7 @@ static int ram_load_postcopy(QEMUFile *f) trace_ram_load_postcopy_loop((uint64_t)addr, flags); if (flags & (RAM_SAVE_FLAG_ZERO | RAM_SAVE_FLAG_PAGE | RAM_SAVE_FLAG_COMPRESS_PAGE)) { - block =3D ram_block_from_stream(f, flags); + block =3D ram_block_from_stream(mis, f, flags); if (!block) { ret =3D -EINVAL; break; @@ -3881,6 +3885,7 @@ void colo_flush_ram_cache(void) */ static int ram_load_precopy(QEMUFile *f) { + MigrationIncomingState *mis =3D migration_incoming_get_current(); int flags =3D 0, ret =3D 0, invalid_flags =3D 0, len =3D 0, i =3D 0; /* ADVISE is earlier, it shows the source has the postcopy capability = on */ bool postcopy_advised =3D postcopy_is_advised(); @@ -3919,7 +3924,7 @@ static int ram_load_precopy(QEMUFile *f) =20 if (flags & (RAM_SAVE_FLAG_ZERO | RAM_SAVE_FLAG_PAGE | RAM_SAVE_FLAG_COMPRESS_PAGE | RAM_SAVE_FLAG_XBZRLE)) { - RAMBlock *block =3D ram_block_from_stream(f, flags); + RAMBlock *block =3D ram_block_from_stream(mis, f, flags); =20 host =3D host_from_ram_block_offset(block, addr); /* --=20 2.32.0 From nobody Sat Apr 27 09:25:08 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 164258334322483.00278598750663; Wed, 19 Jan 2022 01:09:03 -0800 (PST) Received: from localhost ([::1]:58052 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nA6xi-0005XT-5f for importer@patchew.org; Wed, 19 Jan 2022 04:09:02 -0500 Received: from eggs.gnu.org ([209.51.188.92]:58994) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nA639-0007Nt-5A for qemu-devel@nongnu.org; Wed, 19 Jan 2022 03:10:44 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]:43517) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nA634-0001ON-Om for qemu-devel@nongnu.org; Wed, 19 Jan 2022 03:10:32 -0500 Received: from mail-wm1-f69.google.com (mail-wm1-f69.google.com [209.85.128.69]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-183-wBJmkcP3O-Ck-U_jsOxTVA-1; Wed, 19 Jan 2022 03:10:27 -0500 Received: by mail-wm1-f69.google.com with SMTP id 20-20020a05600c22d400b00349067fe7b7so789588wmg.5 for ; Wed, 19 Jan 2022 00:10:27 -0800 (PST) Received: from localhost.localdomain ([85.203.46.175]) by smtp.gmail.com with ESMTPSA id a20sm4351606wmb.27.2022.01.19.00.10.22 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Wed, 19 Jan 2022 00:10:25 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1642579829; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=DSNdZ8aqHo5b9ub2U6pUcsHjlxUs+BHNLv0gjEuikZg=; b=NohtlW1JN+CyHg0JsYJxFd4GP03m70F1lYaEuVd5+8t6J6y4ChZbkqngOO+BEUUU+rfBL+ s3jvPAtf2bWyBfJynOV5v8oA7nJOFyay2Sm04DFGbhy3S60yKqiMRyRBz84U67SNdPjNtF rwxDQqcizNyBWcLwHHCj4UbUGkVuorI= X-MC-Unique: wBJmkcP3O-Ck-U_jsOxTVA-1 X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=DSNdZ8aqHo5b9ub2U6pUcsHjlxUs+BHNLv0gjEuikZg=; b=oD62jDGv9c+FeVLk8zk95edn/4To4A5EDBtBktVhUGaySwWbG7IWqTAoT15Yx8gvE0 NJJLqNIR65REhUekrGR8m1woeZZWIKe3Gl3K0pVbfVKbrAkxdZl8/CoDptOk/YCYVoFb I2ktjIK2NscC1qw3Va/y5IO9xRzr3ZewnGT0U+FXIb81jwMyvX75cI3nejzME7cpJ1ZS 9fA+LTRba9hkgEzcD44zJvU/PH+mWg3YQO4/05zoSHnZFLzSbIZKI1r8QDd449L2gdwI F6589V70xHbzjx3X6iJA1JpGS9Kd73MELLhJcACd6N0jFQIXmfUhPmzMxU/8JbValJOV +EIw== X-Gm-Message-State: AOAM533t4DTjIqqrk4CeKP2hAH9wpAAj1F7Z7npz7rLerI9DAHPHFtN6 sLL6NpZixHvghjAW9exXsdZlOdk+WiDPf8WbtWp28zhYIj+iB5kZ96KaS8IuMLuVOkLpZBeAp1m HEBqHfHyO4ly6dDuX/FF/6wlL9VPExV7ThG8FGIXTYEIJ9D6WK2qSRSE02gcb7/WA X-Received: by 2002:a05:6000:1ac7:: with SMTP id i7mr10013931wry.491.1642579826077; Wed, 19 Jan 2022 00:10:26 -0800 (PST) X-Google-Smtp-Source: ABdhPJzP/4YdssIjtQv13BiQBNACQwTgneVzLOjUK9L2P6TkaWV9NKOXTDG+tiObkelqQ3ofZyyR1g== X-Received: by 2002:a05:6000:1ac7:: with SMTP id i7mr10013911wry.491.1642579825787; Wed, 19 Jan 2022 00:10:25 -0800 (PST) From: Peter Xu To: qemu-devel@nongnu.org Subject: [PATCH RFC 11/15] migration: Add pss.postcopy_requested status Date: Wed, 19 Jan 2022 16:09:25 +0800 Message-Id: <20220119080929.39485-12-peterx@redhat.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20220119080929.39485-1-peterx@redhat.com> References: <20220119080929.39485-1-peterx@redhat.com> MIME-Version: 1.0 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=peterx@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=peterx@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -34 X-Spam_score: -3.5 X-Spam_bar: --- X-Spam_report: (-3.5 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.7, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Juan Quintela , "Dr . David Alan Gilbert" , peterx@redhat.com, Leonardo Bras Soares Passos Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZM-MESSAGEID: 1642583345357100001 Content-Type: text/plain; charset="utf-8" This boolean flag shows whether the current page during migration is trigge= red by postcopy or not. Then in ram_save_host_page() and deeper stack we'll be able to have a reference on the priority of this page. Signed-off-by: Peter Xu Reviewed-by: Dr. David Alan Gilbert --- migration/ram.c | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/migration/ram.c b/migration/ram.c index 3a7d943f9c..b7d17613e8 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -400,6 +400,8 @@ struct PageSearchStatus { unsigned long page; /* Set once we wrap around */ bool complete_round; + /* Whether current page is explicitly requested by postcopy */ + bool postcopy_requested; }; typedef struct PageSearchStatus PageSearchStatus; =20 @@ -1480,6 +1482,9 @@ retry: */ static bool find_dirty_block(RAMState *rs, PageSearchStatus *pss, bool *ag= ain) { + /* This is not a postcopy requested page */ + pss->postcopy_requested =3D false; + pss->page =3D migration_bitmap_find_dirty(rs, pss->block, pss->page); if (pss->complete_round && pss->block =3D=3D rs->last_seen_block && pss->page >=3D rs->last_page) { @@ -1971,6 +1976,7 @@ static bool get_queued_page(RAMState *rs, PageSearchS= tatus *pss) * really rare. */ pss->complete_round =3D false; + pss->postcopy_requested =3D true; } =20 return !!block; --=20 2.32.0 From nobody Sat Apr 27 09:25:08 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1642580807536579.7219535471779; Wed, 19 Jan 2022 00:26:47 -0800 (PST) Received: from localhost ([::1]:43592 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nA6Io-0008N0-G8 for importer@patchew.org; Wed, 19 Jan 2022 03:26:46 -0500 Received: from eggs.gnu.org ([209.51.188.92]:59048) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nA63N-0007T6-ET for qemu-devel@nongnu.org; Wed, 19 Jan 2022 03:10:49 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:31992) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nA638-0001Oa-ST for qemu-devel@nongnu.org; Wed, 19 Jan 2022 03:10:37 -0500 Received: from mail-wm1-f72.google.com (mail-wm1-f72.google.com [209.85.128.72]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-452-O5xeOtseOi6m668XGAuMvg-1; Wed, 19 Jan 2022 03:10:32 -0500 Received: by mail-wm1-f72.google.com with SMTP id f7-20020a1cc907000000b0034b63f314ccso1329038wmb.6 for ; Wed, 19 Jan 2022 00:10:31 -0800 (PST) Received: from localhost.localdomain ([85.203.46.175]) by smtp.gmail.com with ESMTPSA id a20sm4351606wmb.27.2022.01.19.00.10.26 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Wed, 19 Jan 2022 00:10:29 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1642579833; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=m6oq1adDfhZSwTXz9bHotu+7SyhrS/0oPNY1GQjeZV4=; b=OCVJzyjdiDiIiEtABGpELyDWv6a9N8S7GTcVt++eFCFfjFOczNAdbRrCfY3Oz8MAbgncZj WGieqapjd2yK9pogzC2S0YreX6k772hINTD92cWy2vp5Xi+71kmI5myovO2qcAh7GQ9Pdl b+krBENv0rdDPR3feQpngT7/DWO89ug= X-MC-Unique: O5xeOtseOi6m668XGAuMvg-1 X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=m6oq1adDfhZSwTXz9bHotu+7SyhrS/0oPNY1GQjeZV4=; b=1a1HzawUTl1dMqJO727CrC4hoa6UYQoBJcyvfJmPAKNTAS4UeG+iPZlYGFiYuqbFdf +0M1wpO6gzW1gm3qzpTPnVfUG2D1BjUDeDHIAjDQIMbdjYAKttOJWeiTLLmuqbAXgT7i aOWCiSucYEmfZC2eWFB/wfcDczhBedKzVQmdKOM1x1I35vQmBF4v1R5uXI5ivoWib9XI QjPe8Nl5YhVg6xLgTnFfNc5a63PmDAYUP89estTvHRMdgO5GB2A3R8EZ7w+Xa7P99dBb EOjNOO9KPxl/ckICORKITyPccKzEmDA3OIo3laEiwwn0RoK1MaweoEUcP0L3DsrCfkYT oEsw== X-Gm-Message-State: AOAM530kpSPGbOfkJB0/Tz4Kryz9JyjSDIGkzaV8q+HbbUOBdhoh5Wsa xCpjB8c1CoWj/cXsJBENQWmR8zAjiJzX/MLZcTq+n8HCLLp1brMwMb0u6trhP0k4gqrlPX6vLmv N9UvtLlIbCIQzxu6WIuU6NffOJT2v2oDS8iUWR4ZiWk8w8JX74Ga9nz5pNZ43rfmo X-Received: by 2002:a05:600c:28cc:: with SMTP id h12mr2303618wmd.18.1642579830131; Wed, 19 Jan 2022 00:10:30 -0800 (PST) X-Google-Smtp-Source: ABdhPJzBuPn0xaYRF5x/f76lglgXVjY6Z7eL9CB/h4lLV85Zi1zgMs0Tq53xC5qUWw1NAsO+AaYrmw== X-Received: by 2002:a05:600c:28cc:: with SMTP id h12mr2303594wmd.18.1642579829761; Wed, 19 Jan 2022 00:10:29 -0800 (PST) From: Peter Xu To: qemu-devel@nongnu.org Subject: [PATCH RFC 12/15] migration: Move migrate_allow_multifd and helpers into migration.c Date: Wed, 19 Jan 2022 16:09:26 +0800 Message-Id: <20220119080929.39485-13-peterx@redhat.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20220119080929.39485-1-peterx@redhat.com> References: <20220119080929.39485-1-peterx@redhat.com> MIME-Version: 1.0 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=peterx@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=peterx@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -34 X-Spam_score: -3.5 X-Spam_bar: --- X-Spam_report: (-3.5 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.7, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Juan Quintela , "Dr . David Alan Gilbert" , peterx@redhat.com, Leonardo Bras Soares Passos Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZM-MESSAGEID: 1642580809343100001 Content-Type: text/plain; charset="utf-8" This variable, along with its helpers, is used to detect whether multiple channel will be supported for migration. In follow up patches, there'll be other capability that requires multi-channels. Hence move it outside multi= fd specific code and make it public. Meanwhile rename it from "multifd" to "multi_channels" to show its real meaning. Signed-off-by: Peter Xu Reviewed-by: Dr. David Alan Gilbert --- migration/migration.c | 22 +++++++++++++++++----- migration/migration.h | 3 +++ migration/multifd.c | 19 ++++--------------- migration/multifd.h | 2 -- 4 files changed, 24 insertions(+), 22 deletions(-) diff --git a/migration/migration.c b/migration/migration.c index 252ce1eaec..15a48b548a 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -180,6 +180,18 @@ static int migration_maybe_pause(MigrationState *s, int new_state); static void migrate_fd_cancel(MigrationState *s); =20 +static bool migrate_allow_multi_channels =3D true; + +void migrate_protocol_allow_multi_channels(bool allow) +{ + migrate_allow_multi_channels =3D allow; +} + +bool migrate_multi_channels_is_allowed(void) +{ + return migrate_allow_multi_channels; +} + static gint page_request_addr_cmp(gconstpointer ap, gconstpointer bp) { uintptr_t a =3D (uintptr_t) ap, b =3D (uintptr_t) bp; @@ -463,12 +475,12 @@ static void qemu_start_incoming_migration(const char = *uri, Error **errp) { const char *p =3D NULL; =20 - migrate_protocol_allow_multifd(false); /* reset it anyway */ + migrate_protocol_allow_multi_channels(false); /* reset it anyway */ qapi_event_send_migration(MIGRATION_STATUS_SETUP); if (strstart(uri, "tcp:", &p) || strstart(uri, "unix:", NULL) || strstart(uri, "vsock:", NULL)) { - migrate_protocol_allow_multifd(true); + migrate_protocol_allow_multi_channels(true); socket_start_incoming_migration(p ? p : uri, errp); #ifdef CONFIG_RDMA } else if (strstart(uri, "rdma:", &p)) { @@ -1252,7 +1264,7 @@ static bool migrate_caps_check(bool *cap_list, =20 /* incoming side only */ if (runstate_check(RUN_STATE_INMIGRATE) && - !migrate_multifd_is_allowed() && + !migrate_multi_channels_is_allowed() && cap_list[MIGRATION_CAPABILITY_MULTIFD]) { error_setg(errp, "multifd is not supported by current protocol"); return false; @@ -2310,11 +2322,11 @@ void qmp_migrate(const char *uri, bool has_blk, boo= l blk, } } =20 - migrate_protocol_allow_multifd(false); + migrate_protocol_allow_multi_channels(false); if (strstart(uri, "tcp:", &p) || strstart(uri, "unix:", NULL) || strstart(uri, "vsock:", NULL)) { - migrate_protocol_allow_multifd(true); + migrate_protocol_allow_multi_channels(true); socket_start_outgoing_migration(s, p ? p : uri, &local_err); #ifdef CONFIG_RDMA } else if (strstart(uri, "rdma:", &p)) { diff --git a/migration/migration.h b/migration/migration.h index 34b79cb961..d0c0902ec9 100644 --- a/migration/migration.h +++ b/migration/migration.h @@ -425,4 +425,7 @@ void migration_cancel(const Error *error); =20 void populate_vfio_info(MigrationInfo *info); =20 +bool migrate_multi_channels_is_allowed(void); +void migrate_protocol_allow_multi_channels(bool allow); + #endif diff --git a/migration/multifd.c b/migration/multifd.c index 3242f688e5..64ca50de62 100644 --- a/migration/multifd.c +++ b/migration/multifd.c @@ -535,7 +535,7 @@ void multifd_save_cleanup(void) { int i; =20 - if (!migrate_use_multifd() || !migrate_multifd_is_allowed()) { + if (!migrate_use_multifd() || !migrate_multi_channels_is_allowed()) { return; } multifd_send_terminate_threads(NULL); @@ -870,17 +870,6 @@ cleanup: multifd_new_send_channel_cleanup(p, sioc, local_err); } =20 -static bool migrate_allow_multifd =3D true; -void migrate_protocol_allow_multifd(bool allow) -{ - migrate_allow_multifd =3D allow; -} - -bool migrate_multifd_is_allowed(void) -{ - return migrate_allow_multifd; -} - int multifd_save_setup(Error **errp) { int thread_count; @@ -891,7 +880,7 @@ int multifd_save_setup(Error **errp) if (!migrate_use_multifd()) { return 0; } - if (!migrate_multifd_is_allowed()) { + if (!migrate_multi_channels_is_allowed()) { error_setg(errp, "multifd is not supported by current protocol"); return -1; } @@ -989,7 +978,7 @@ int multifd_load_cleanup(Error **errp) { int i; =20 - if (!migrate_use_multifd() || !migrate_multifd_is_allowed()) { + if (!migrate_use_multifd() || !migrate_multi_channels_is_allowed()) { return 0; } multifd_recv_terminate_threads(NULL); @@ -1138,7 +1127,7 @@ int multifd_load_setup(Error **errp) if (!migrate_use_multifd()) { return 0; } - if (!migrate_multifd_is_allowed()) { + if (!migrate_multi_channels_is_allowed()) { error_setg(errp, "multifd is not supported by current protocol"); return -1; } diff --git a/migration/multifd.h b/migration/multifd.h index e57adc783b..0ed07794b6 100644 --- a/migration/multifd.h +++ b/migration/multifd.h @@ -13,8 +13,6 @@ #ifndef QEMU_MIGRATION_MULTIFD_H #define QEMU_MIGRATION_MULTIFD_H =20 -bool migrate_multifd_is_allowed(void); -void migrate_protocol_allow_multifd(bool allow); int multifd_save_setup(Error **errp); void multifd_save_cleanup(void); int multifd_load_setup(Error **errp); --=20 2.32.0 From nobody Sat Apr 27 09:25:08 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1642583550270466.8542108855804; Wed, 19 Jan 2022 01:12:30 -0800 (PST) Received: from localhost ([::1]:37662 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nA712-0002Ve-Nj for importer@patchew.org; Wed, 19 Jan 2022 04:12:28 -0500 Received: from eggs.gnu.org ([209.51.188.92]:59080) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nA63O-0007Vg-Vm for qemu-devel@nongnu.org; Wed, 19 Jan 2022 03:10:51 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]:26112) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nA63E-0001P8-3i for qemu-devel@nongnu.org; Wed, 19 Jan 2022 03:10:50 -0500 Received: from mail-wm1-f72.google.com (mail-wm1-f72.google.com [209.85.128.72]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-437-BVCKtXXgOnKu25v5sFsBPw-1; Wed, 19 Jan 2022 03:10:35 -0500 Received: by mail-wm1-f72.google.com with SMTP id s190-20020a1ca9c7000000b00347c6c39d9aso1332147wme.5 for ; Wed, 19 Jan 2022 00:10:35 -0800 (PST) Received: from localhost.localdomain ([85.203.46.175]) by smtp.gmail.com with ESMTPSA id a20sm4351606wmb.27.2022.01.19.00.10.30 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Wed, 19 Jan 2022 00:10:33 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1642579836; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=pofGhg+WExSJvEPG5CZzSLd7oFU6LQnPy1cjEzM5ecA=; b=YKkdjRFh1vAk+YRFqbRk4WsRSUDc9PivUvujBcnTyXUjPtroRgUS5ZReXysbiWukV4ZnQh lRPHmYgOiyuPIPVKUHrZA7iWD5JVO9EE/vxLavf8Fko6LcCSvAoE4lzDBogHIegthESCqV st/unJ4n2XKKwcet9JG3Ho6RB4R8MhY= X-MC-Unique: BVCKtXXgOnKu25v5sFsBPw-1 X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=pofGhg+WExSJvEPG5CZzSLd7oFU6LQnPy1cjEzM5ecA=; b=BydlyQiS5qFR/1yOGB2iFyGB+nNS1RO5MTo0K0EKlaBigF1vwK4qx2duEklvUcb/gb 4YTrJzRxIGiut5wP8I4zE1H0Nj2uUFsPJUiwxtvH1U3otVNQQTO5NHbyL+HzyAmn9Vlc n5wSNQ93GNetl6ktrlP3qyFEbdbN10IpUX8xhBEQskCexSkzKUe+BTCdRXp5+3dx7X/y 8qdH8ciKqJ1hAHYQskiS1YvS0IEUxqRS6ehNtJFCFWX2W851E25SSzVp3H756QRxObmL dAMIApzu1fD2FM4Ju+1AUNfJuz4hTIT8FfExK6/pHlys8EiWD3eBjok18W3349CgdpYU S5Vw== X-Gm-Message-State: AOAM531tTxSS1AdePfLiQZ2cLnmjAiDoClJjbfdiROBq0XM+iowuLwqV 23gQhjXbA5Rv6sqib+tBCOGGvTP4SHnmBCcXNCdXvWB95LscI+DaJ0u/Fv+aVgQFTf/8ft+2r5k G0PA5ZiB2k7TQqN/6hoOhJeeLRjJmR8Q5fdK3crBjgkHvlS8n98MWSGVPKzMWj4y+ X-Received: by 2002:adf:fd4e:: with SMTP id h14mr1842784wrs.404.1642579834013; Wed, 19 Jan 2022 00:10:34 -0800 (PST) X-Google-Smtp-Source: ABdhPJzr479xY7XIvPHCGyHtzDZZymn0aLPVUIljMhbw/pdWZ1iI9i3ot28lFgwSwYxxQisYNd4x+A== X-Received: by 2002:adf:fd4e:: with SMTP id h14mr1842757wrs.404.1642579833705; Wed, 19 Jan 2022 00:10:33 -0800 (PST) From: Peter Xu To: qemu-devel@nongnu.org Subject: [PATCH RFC 13/15] migration: Add postcopy-preempt capability Date: Wed, 19 Jan 2022 16:09:27 +0800 Message-Id: <20220119080929.39485-14-peterx@redhat.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20220119080929.39485-1-peterx@redhat.com> References: <20220119080929.39485-1-peterx@redhat.com> MIME-Version: 1.0 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=peterx@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=peterx@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -34 X-Spam_score: -3.5 X-Spam_bar: --- X-Spam_report: (-3.5 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.7, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Juan Quintela , "Dr . David Alan Gilbert" , peterx@redhat.com, Leonardo Bras Soares Passos Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZM-MESSAGEID: 1642583551442100001 Content-Type: text/plain; charset="utf-8" Firstly, postcopy already preempts precopy due to the fact that we do unqueue_page() first before looking into dirty bits. However that's not enough, e.g., when there're host huge page enabled, when sending a precopy huge page, a postcopy request needs to wait until the who= le huge page that is sending to finish. That could introduce quite some delay, the bigger the huge page is the larger delay it'll bring. This patch adds a new capability to allow postcopy requests to preempt exis= ting precopy page during sending a huge page, so that postcopy requests can be serviced even faster. Meanwhile to send it even faster, bypass the precopy stream by providing a standalone postcopy socket for sending requested pages. Since the new behavior will not be compatible with the old behavior, this w= ill not be the default, it's enabled only when the new capability is set on both src/dst QEMUs. This patch only adds the capability itself, the logic will be added in foll= ow up patches. Signed-off-by: Peter Xu Reviewed-by: Dr. David Alan Gilbert --- migration/migration.c | 23 +++++++++++++++++++++++ migration/migration.h | 1 + qapi/migration.json | 8 +++++++- 3 files changed, 31 insertions(+), 1 deletion(-) diff --git a/migration/migration.c b/migration/migration.c index 15a48b548a..84a8fbd80d 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -1227,6 +1227,11 @@ static bool migrate_caps_check(bool *cap_list, error_setg(errp, "Postcopy is not compatible with ignore-share= d"); return false; } + + if (cap_list[MIGRATION_CAPABILITY_MULTIFD]) { + error_setg(errp, "Multifd is not supported in postcopy"); + return false; + } } =20 if (cap_list[MIGRATION_CAPABILITY_BACKGROUND_SNAPSHOT]) { @@ -1270,6 +1275,13 @@ static bool migrate_caps_check(bool *cap_list, return false; } =20 + if (cap_list[MIGRATION_CAPABILITY_POSTCOPY_PREEMPT]) { + if (!cap_list[MIGRATION_CAPABILITY_POSTCOPY_RAM]) { + error_setg(errp, "Postcopy preempt requires postcopy-ram"); + return false; + } + } + return true; } =20 @@ -2623,6 +2635,15 @@ bool migrate_background_snapshot(void) return s->enabled_capabilities[MIGRATION_CAPABILITY_BACKGROUND_SNAPSHO= T]; } =20 +bool migrate_postcopy_preempt(void) +{ + MigrationState *s; + + s =3D migrate_get_current(); + + return s->enabled_capabilities[MIGRATION_CAPABILITY_POSTCOPY_PREEMPT]; +} + /* migration thread support */ /* * Something bad happened to the RP stream, mark an error @@ -4239,6 +4260,8 @@ static Property migration_properties[] =3D { DEFINE_PROP_MIG_CAP("x-compress", MIGRATION_CAPABILITY_COMPRESS), DEFINE_PROP_MIG_CAP("x-events", MIGRATION_CAPABILITY_EVENTS), DEFINE_PROP_MIG_CAP("x-postcopy-ram", MIGRATION_CAPABILITY_POSTCOPY_RA= M), + DEFINE_PROP_MIG_CAP("x-postcopy-preempt", + MIGRATION_CAPABILITY_POSTCOPY_PREEMPT), DEFINE_PROP_MIG_CAP("x-colo", MIGRATION_CAPABILITY_X_COLO), DEFINE_PROP_MIG_CAP("x-release-ram", MIGRATION_CAPABILITY_RELEASE_RAM), DEFINE_PROP_MIG_CAP("x-block", MIGRATION_CAPABILITY_BLOCK), diff --git a/migration/migration.h b/migration/migration.h index d0c0902ec9..9d39ccfcf5 100644 --- a/migration/migration.h +++ b/migration/migration.h @@ -391,6 +391,7 @@ int migrate_decompress_threads(void); bool migrate_use_events(void); bool migrate_postcopy_blocktime(void); bool migrate_background_snapshot(void); +bool migrate_postcopy_preempt(void); =20 /* Sending on the return path - generic and then for each message type */ void migrate_send_rp_shut(MigrationIncomingState *mis, diff --git a/qapi/migration.json b/qapi/migration.json index bbfd48cf0b..f00b365bd5 100644 --- a/qapi/migration.json +++ b/qapi/migration.json @@ -452,6 +452,12 @@ # procedure starts. The VM RAM is saved with running= VM. # (since 6.0) # +# @postcopy-preempt: If enabled, the migration process will allow postcopy +# requests to preempt precopy stream, so postcopy reque= sts +# will be handled faster. This is a performance featur= e and +# should not affect the correctness of postcopy migrati= on. +# (since 7.0) +# # Features: # @unstable: Members @x-colo and @x-ignore-shared are experimental. # @@ -465,7 +471,7 @@ 'block', 'return-path', 'pause-before-switchover', 'multifd', 'dirty-bitmaps', 'postcopy-blocktime', 'late-block-activate', { 'name': 'x-ignore-shared', 'features': [ 'unstable' ] }, - 'validate-uuid', 'background-snapshot'] } + 'validate-uuid', 'background-snapshot', 'postcopy-preempt'] } =20 ## # @MigrationCapabilityStatus: --=20 2.32.0 From nobody Sat Apr 27 09:25:08 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1642581465467458.6246326212571; Wed, 19 Jan 2022 00:37:45 -0800 (PST) Received: from localhost ([::1]:55206 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nA6TQ-0008Hy-87 for importer@patchew.org; Wed, 19 Jan 2022 03:37:44 -0500 Received: from eggs.gnu.org ([209.51.188.92]:59162) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nA63Q-0007Xt-N4 for qemu-devel@nongnu.org; Wed, 19 Jan 2022 03:10:52 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:33416) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nA63N-0001PU-54 for qemu-devel@nongnu.org; Wed, 19 Jan 2022 03:10:52 -0500 Received: from mail-wm1-f69.google.com (mail-wm1-f69.google.com [209.85.128.69]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-512-g81Pft65PQ2rHuRmbOK1vw-1; Wed, 19 Jan 2022 03:10:40 -0500 Received: by mail-wm1-f69.google.com with SMTP id c62-20020a1c9a41000000b0034da9d62199so604426wme.7 for ; Wed, 19 Jan 2022 00:10:40 -0800 (PST) Received: from localhost.localdomain ([85.203.46.175]) by smtp.gmail.com with ESMTPSA id a20sm4351606wmb.27.2022.01.19.00.10.34 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Wed, 19 Jan 2022 00:10:37 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1642579842; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=tvxtDnSFugefeJRTXG50co54t5xlvxJzvdP+AwPfn6M=; b=EBzTKyzYei1CuIYb6sHWWNI4PQzjyvd4pqdmRo8TJU9IPx+njUi3o4vRcSO8FlEGaJF10r /niL0n+90KU8aSmXx8zvQcx77apd5kMI7TRxfRwD5dJi5oWQ0oiqzMmeSwKQsyheuz4miu sw/dzczZiYNmiUSRJ0NaVWyfecruwhI= X-MC-Unique: g81Pft65PQ2rHuRmbOK1vw-1 X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=tvxtDnSFugefeJRTXG50co54t5xlvxJzvdP+AwPfn6M=; b=73rhoDlKhZ1hYmi2daF9EZZ5QNiSq1W089eqsBbwdfrrf35ml2OvJ07xzOv3NKg6RB ZfBkjkRD6L6V49KJJJy5VzFd+dDd/Ur4B9s26NK3eXfVJu65kA8thRQU9dB4QDoCzq4Z 88MqurbTpWHZ6iOC4HjuLiVmcxd7M2QT99PljA884JFac0SS9FT4mt/bQ9sOVe2zbT22 UOkXzHaaxqY9X00HJK0p2NpA+PIF0qnGfai9wNdeXtiu5Cwyfi+mMbx0zeV5Rjzx/fsm fF6LeMZlvm8M2uhP80iAmf3tZYjDv8jWmFHtzAzAjfupl13IYtxzRU8KiOaTTLHc4h82 NRuA== X-Gm-Message-State: AOAM531NJ9R0ImsXk8s2l+FkYoBHGtPZEmUFgbzpFHkvySn/Q3OAn7KA zsDPns7CCmSTzvS4HJjOH5GGGdBikbeWooaYpmhi1ANBbD4LFYiewOVxlwFCMsqmJAOZvma3NFw pVtjlJdzNGGQWmbaIRz/Boh+FowEaygFrLJ35/7oMVNdkKPlHQ/lMfyudsZ4rSYoy X-Received: by 2002:a05:6000:144f:: with SMTP id v15mr6624607wrx.514.1642579838724; Wed, 19 Jan 2022 00:10:38 -0800 (PST) X-Google-Smtp-Source: ABdhPJzqN8JMG/bAanfSVUvEF4KcIhbxKKLk7PMjVdQ340e0vFUTyXHt2ZS6QPCshadBEKog7C/T/A== X-Received: by 2002:a05:6000:144f:: with SMTP id v15mr6624552wrx.514.1642579837968; Wed, 19 Jan 2022 00:10:37 -0800 (PST) From: Peter Xu To: qemu-devel@nongnu.org Subject: [PATCH RFC 14/15] migration: Postcopy preemption on separate channel Date: Wed, 19 Jan 2022 16:09:28 +0800 Message-Id: <20220119080929.39485-15-peterx@redhat.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20220119080929.39485-1-peterx@redhat.com> References: <20220119080929.39485-1-peterx@redhat.com> MIME-Version: 1.0 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=peterx@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=peterx@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -28 X-Spam_score: -2.9 X-Spam_bar: -- X-Spam_report: (-2.9 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.7, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URG_BIZ=0.573 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Juan Quintela , "Dr . David Alan Gilbert" , peterx@redhat.com, Leonardo Bras Soares Passos Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZM-MESSAGEID: 1642581467150100001 Content-Type: text/plain; charset="utf-8" This patch enables postcopy-preempt feature. It contains two major changes to the migration logic: (1) Postcopy requests are now sent via a different socket from precopy background migration stream, so as to be isolated from very high page request delays (2) For huge page enabled hosts: when there's postcopy requests, they can= now intercept a partial sending of huge host pages on src QEMU. After this patch, we'll have two "channels" (or say, sockets, because it's = only supported on socket-based channels) for postcopy: (1) PRECOPY channel (whic= h is the default channel that transfers background pages), and (2) POSTCOPY channel (which only transfers requested pages). On the source QEMU, when we found a postcopy request, we'll interrupt the PRECOPY channel sending process and quickly switch to the POSTCOPY channel. After we serviced all the high priority postcopy pages, we'll switch back to PRECOPY channel so that we'll continue to send the interrupted huge page ag= ain. There's no new thread introduced. On the destination QEMU, one new thread is introduced to receive page data = from the postcopy specific socket. This patch has a side effect. After sending postcopy pages, previously we'= ll assume the guest will access follow up pages so we'll keep sending from the= re. Now it's changed. Instead of going on with a postcopy requested page, we'l= l go back and continue sending the precopy huge page (which can be intercepted b= y a postcopy request so the huge page can be sent partially before). Whether that's a problem is debatable, because "assuming the guest will continue to access the next page" doesn't really suite when huge pages are used, especially if the huge page is large (e.g. 1GB pages). So that local= ity hint is much meaningless if huge pages are used. If postcopy preempt is enabled, a separate channel is created for it so tha= t it can be used later for postcopy specific page requests. On dst node, a standalone thread is used to receive postcopy requested pages. The thread = is created along with the ram listen thread during POSTCOPY_LISTEN phase. Signed-off-by: Peter Xu --- migration/migration.c | 62 +++++++-- migration/migration.h | 10 +- migration/postcopy-ram.c | 65 ++++++++- migration/postcopy-ram.h | 10 ++ migration/ram.c | 294 +++++++++++++++++++++++++++++++++++++-- migration/ram.h | 2 + migration/socket.c | 18 +++ migration/socket.h | 1 + migration/trace-events | 10 ++ 9 files changed, 445 insertions(+), 27 deletions(-) diff --git a/migration/migration.c b/migration/migration.c index 84a8fbd80d..13dc6ecd37 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -315,6 +315,12 @@ void migration_incoming_state_destroy(void) mis->socket_address_list =3D NULL; } =20 + if (mis->postcopy_qemufile_dst) { + migration_ioc_unregister_yank_from_file(mis->postcopy_qemufile_dst= ); + qemu_fclose(mis->postcopy_qemufile_dst); + mis->postcopy_qemufile_dst =3D NULL; + } + yank_unregister_instance(MIGRATION_YANK_INSTANCE); } =20 @@ -708,15 +714,21 @@ void migration_fd_process_incoming(QEMUFile *f, Error= **errp) migration_incoming_process(); } =20 +static bool migration_needs_multiple_sockets(void) +{ + return migrate_use_multifd() || migrate_postcopy_preempt(); +} + void migration_ioc_process_incoming(QIOChannel *ioc, Error **errp) { MigrationIncomingState *mis =3D migration_incoming_get_current(); Error *local_err =3D NULL; bool start_migration; + QEMUFile *f; =20 if (!mis->from_src_file) { /* The first connection (multifd may have multiple) */ - QEMUFile *f =3D qemu_fopen_channel_input(ioc); + f =3D qemu_fopen_channel_input(ioc); =20 /* If it's a recovery, we're done */ if (postcopy_try_recover(f)) { @@ -729,13 +741,18 @@ void migration_ioc_process_incoming(QIOChannel *ioc, = Error **errp) =20 /* * Common migration only needs one channel, so we can start - * right now. Multifd needs more than one channel, we wait. + * right now. Some features need more than one channel, we wait. */ - start_migration =3D !migrate_use_multifd(); + start_migration =3D !migration_needs_multiple_sockets(); } else { /* Multiple connections */ - assert(migrate_use_multifd()); - start_migration =3D multifd_recv_new_channel(ioc, &local_err); + assert(migration_needs_multiple_sockets()); + if (migrate_use_multifd()) { + start_migration =3D multifd_recv_new_channel(ioc, &local_err); + } else if (migrate_postcopy_preempt()) { + f =3D qemu_fopen_channel_input(ioc); + start_migration =3D postcopy_preempt_new_channel(mis, f); + } if (local_err) { error_propagate(errp, local_err); return; @@ -756,11 +773,20 @@ void migration_ioc_process_incoming(QIOChannel *ioc, = Error **errp) bool migration_has_all_channels(void) { MigrationIncomingState *mis =3D migration_incoming_get_current(); - bool all_channels; =20 - all_channels =3D multifd_recv_all_channels_created(); + if (!mis->from_src_file) { + return false; + } + + if (migrate_use_multifd()) { + return multifd_recv_all_channels_created(); + } + + if (migrate_postcopy_preempt()) { + return mis->postcopy_qemufile_dst !=3D NULL; + } =20 - return all_channels && mis->from_src_file !=3D NULL; + return true; } =20 /* @@ -1850,6 +1876,11 @@ static void migrate_fd_cleanup(MigrationState *s) qemu_fclose(tmp); } =20 + if (s->postcopy_qemufile_src) { + qemu_fclose(s->postcopy_qemufile_src); + s->postcopy_qemufile_src =3D NULL; + } + assert(!migration_is_active(s)); =20 if (s->state =3D=3D MIGRATION_STATUS_CANCELLING) { @@ -3122,6 +3153,8 @@ static int postcopy_start(MigrationState *ms) MIGRATION_STATUS_FAILED); } =20 + trace_postcopy_preempt_enabled(migrate_postcopy_preempt()); + return ret; =20 fail_closefb: @@ -3234,6 +3267,11 @@ static void migration_completion(MigrationState *s) qemu_savevm_state_complete_postcopy(s->to_dst_file); qemu_mutex_unlock_iothread(); =20 + /* Shutdown the postcopy fast path thread */ + if (migrate_postcopy_preempt()) { + postcopy_preempt_shutdown_file(s); + } + trace_migration_completion_postcopy_end_after_complete(); } else if (s->state =3D=3D MIGRATION_STATUS_CANCELLING) { goto fail; @@ -4143,6 +4181,14 @@ void migrate_fd_connect(MigrationState *s, Error *er= ror_in) return; } =20 + if (postcopy_preempt_setup(s, &local_err)) { + error_report_err(local_err); + migrate_set_state(&s->state, MIGRATION_STATUS_SETUP, + MIGRATION_STATUS_FAILED); + migrate_fd_cleanup(s); + return; + } + if (migrate_background_snapshot()) { qemu_thread_create(&s->thread, "bg_snapshot", bg_migration_thread, s, QEMU_THREAD_JOINABLE); diff --git a/migration/migration.h b/migration/migration.h index 9d39ccfcf5..8786785b1f 100644 --- a/migration/migration.h +++ b/migration/migration.h @@ -23,6 +23,7 @@ #include "io/channel-buffer.h" #include "net/announce.h" #include "qom/object.h" +#include "postcopy-ram.h" =20 struct PostcopyBlocktimeContext; =20 @@ -67,7 +68,7 @@ typedef struct { struct MigrationIncomingState { QEMUFile *from_src_file; /* Previously received RAM's RAMBlock pointer */ - RAMBlock *last_recv_block; + RAMBlock *last_recv_block[RAM_CHANNEL_MAX]; /* A hook to allow cleanup at the end of incoming migration */ void *transport_data; void (*transport_cleanup)(void *data); @@ -109,6 +110,11 @@ struct MigrationIncomingState { * enabled. */ int postcopy_channels; + /* QEMUFile for postcopy only; it'll be handled by a separate thread */ + QEMUFile *postcopy_qemufile_dst; + /* Postcopy priority thread is used to receive postcopy requested page= s */ + QemuThread postcopy_prio_thread; + bool postcopy_prio_thread_created; /* * An array of temp host huge pages to be used, one for each postcopy * channel. @@ -189,6 +195,8 @@ struct MigrationState { QEMUBH *cleanup_bh; /* Protected by qemu_file_lock */ QEMUFile *to_dst_file; + /* Postcopy specific transfer channel */ + QEMUFile *postcopy_qemufile_src; QIOChannelBuffer *bioc; /* * Protects to_dst_file/from_dst_file pointers. We need to make sure = we diff --git a/migration/postcopy-ram.c b/migration/postcopy-ram.c index 88c832eeba..9006e68fd1 100644 --- a/migration/postcopy-ram.c +++ b/migration/postcopy-ram.c @@ -32,6 +32,8 @@ #include "trace.h" #include "hw/boards.h" #include "exec/ramblock.h" +#include "socket.h" +#include "qemu-file-channel.h" =20 /* Arbitrary limit on size of each discard command, * keeps them around ~200 bytes @@ -562,6 +564,11 @@ int postcopy_ram_incoming_cleanup(MigrationIncomingSta= te *mis) { trace_postcopy_ram_incoming_cleanup_entry(); =20 + if (mis->postcopy_prio_thread_created) { + qemu_thread_join(&mis->postcopy_prio_thread); + mis->postcopy_prio_thread_created =3D false; + } + if (mis->have_fault_thread) { Error *local_err =3D NULL; =20 @@ -1114,8 +1121,13 @@ static int postcopy_temp_pages_setup(MigrationIncomi= ngState *mis) int err, i, channels; void *temp_page; =20 - /* TODO: will be boosted when enable postcopy preemption */ - mis->postcopy_channels =3D 1; + if (migrate_postcopy_preempt()) { + /* If preemption enabled, need extra channel for urgent requests */ + mis->postcopy_channels =3D RAM_CHANNEL_MAX; + } else { + /* Both precopy/postcopy on the same channel */ + mis->postcopy_channels =3D 1; + } =20 channels =3D mis->postcopy_channels; mis->postcopy_tmp_pages =3D g_malloc0(sizeof(PostcopyTmpPage) * channe= ls); @@ -1182,7 +1194,7 @@ int postcopy_ram_incoming_setup(MigrationIncomingStat= e *mis) return -1; } =20 - postcopy_thread_create(mis, &mis->fault_thread, "postcopy/fault", + postcopy_thread_create(mis, &mis->fault_thread, "qemu/fault-default", postcopy_ram_fault_thread, QEMU_THREAD_JOINABLE= ); mis->have_fault_thread =3D true; =20 @@ -1197,6 +1209,16 @@ int postcopy_ram_incoming_setup(MigrationIncomingSta= te *mis) return -1; } =20 + if (migrate_postcopy_preempt()) { + /* + * This thread needs to be created after the temp pages because it= 'll fetch + * RAM_CHANNEL_POSTCOPY PostcopyTmpPage immediately. + */ + postcopy_thread_create(mis, &mis->postcopy_prio_thread, "qemu/faul= t-fast", + postcopy_preempt_thread, QEMU_THREAD_JOINAB= LE); + mis->postcopy_prio_thread_created =3D true; + } + trace_postcopy_ram_enable_notify(); =20 return 0; @@ -1516,3 +1538,40 @@ void postcopy_unregister_shared_ufd(struct PostCopyF= D *pcfd) } } } + +bool postcopy_preempt_new_channel(MigrationIncomingState *mis, QEMUFile *f= ile) +{ + mis->postcopy_qemufile_dst =3D file; + + trace_postcopy_preempt_new_channel(); + + /* Start the migration immediately */ + return true; +} + +int postcopy_preempt_setup(MigrationState *s, Error **errp) +{ + QIOChannel *ioc; + + if (!migrate_postcopy_preempt()) { + return 0; + } + + if (!migrate_multi_channels_is_allowed()) { + error_setg(errp, "Postcopy preempt is not supported as current " + "migration stream does not support multi-channels."); + return -1; + } + + ioc =3D socket_send_channel_create_sync(errp); + + if (ioc =3D=3D NULL) { + return -1; + } + + s->postcopy_qemufile_src =3D qemu_fopen_channel_output(ioc); + + trace_postcopy_preempt_new_channel(); + + return 0; +} diff --git a/migration/postcopy-ram.h b/migration/postcopy-ram.h index 07684c0e1d..34b1080cde 100644 --- a/migration/postcopy-ram.h +++ b/migration/postcopy-ram.h @@ -183,4 +183,14 @@ int postcopy_wake_shared(struct PostCopyFD *pcfd, uint= 64_t client_addr, int postcopy_request_shared_page(struct PostCopyFD *pcfd, RAMBlock *rb, uint64_t client_addr, uint64_t offset); =20 +/* Hard-code channels for now for postcopy preemption */ +enum PostcopyChannels { + RAM_CHANNEL_PRECOPY =3D 0, + RAM_CHANNEL_POSTCOPY =3D 1, + RAM_CHANNEL_MAX, +}; + +bool postcopy_preempt_new_channel(MigrationIncomingState *mis, QEMUFile *f= ile); +int postcopy_preempt_setup(MigrationState *s, Error **errp); + #endif diff --git a/migration/ram.c b/migration/ram.c index b7d17613e8..6a1ef86eca 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -294,6 +294,20 @@ struct RAMSrcPageRequest { QSIMPLEQ_ENTRY(RAMSrcPageRequest) next_req; }; =20 +typedef struct { + /* + * Cached ramblock/offset values if preempted. They're only meaningfu= l if + * preempted=3D=3Dtrue below. + */ + RAMBlock *ram_block; + unsigned long ram_page; + /* + * Whether a postcopy preemption just happened. Will be reset after + * precopy recovered to background migration. + */ + bool preempted; +} PostcopyPreemptState; + /* State of RAM for migration */ struct RAMState { /* QEMUFile used for this migration */ @@ -347,6 +361,14 @@ struct RAMState { /* Queue of outstanding page requests from the destination */ QemuMutex src_page_req_mutex; QSIMPLEQ_HEAD(, RAMSrcPageRequest) src_page_requests; + + /* Postcopy preemption informations */ + PostcopyPreemptState postcopy_preempt_state; + /* + * Current channel we're using on src VM. Only valid if postcopy-pree= mpt + * is enabled. + */ + int postcopy_channel; }; typedef struct RAMState RAMState; =20 @@ -354,6 +376,11 @@ static RAMState *ram_state; =20 static NotifierWithReturnList precopy_notifier_list; =20 +static void postcopy_preempt_reset(RAMState *rs) +{ + memset(&rs->postcopy_preempt_state, 0, sizeof(PostcopyPreemptState)); +} + /* Whether postcopy has queued requests? */ static bool postcopy_has_request(RAMState *rs) { @@ -1937,6 +1964,55 @@ void ram_write_tracking_stop(void) } #endif /* defined(__linux__) */ =20 +/* + * Check whether two addr/offset of the ramblock falls onto the same host = huge + * page. Returns true if so, false otherwise. + */ +static bool offset_on_same_huge_page(RAMBlock *rb, uint64_t addr1, + uint64_t addr2) +{ + size_t page_size =3D qemu_ram_pagesize(rb); + + addr1 =3D ROUND_DOWN(addr1, page_size); + addr2 =3D ROUND_DOWN(addr2, page_size); + + return addr1 =3D=3D addr2; +} + +/* + * Whether a previous preempted precopy huge page contains current request= ed + * page? Returns true if so, false otherwise. + * + * This should really happen very rarely, because it means when we were se= nding + * during background migration for postcopy we're sending exactly the page= that + * some vcpu got faulted on on dest node. When it happens, we probably do= n't + * need to do much but drop the request, because we know right after we re= store + * the precopy stream it'll be serviced. It'll slightly affect the order = of + * postcopy requests to be serviced (e.g. it'll be the same as we move cur= rent + * request to the end of the queue) but it shouldn't be a big deal. The m= ost + * imporant thing is we can _never_ try to send a partial-sent huge page o= n the + * POSTCOPY channel again, otherwise that huge page will got "split brain"= on + * two channels (PRECOPY, POSTCOPY). + */ +static bool postcopy_preempted_contains(RAMState *rs, RAMBlock *block, + ram_addr_t offset) +{ + PostcopyPreemptState *state =3D &rs->postcopy_preempt_state; + + /* No preemption at all? */ + if (!state->preempted) { + return false; + } + + /* Not even the same ramblock? */ + if (state->ram_block !=3D block) { + return false; + } + + return offset_on_same_huge_page(block, offset, + state->ram_page << TARGET_PAGE_BITS); +} + /** * get_queued_page: unqueue a page from the postcopy requests * @@ -1952,9 +2028,17 @@ static bool get_queued_page(RAMState *rs, PageSearch= Status *pss) RAMBlock *block; ram_addr_t offset; =20 +again: block =3D unqueue_page(rs, &offset); =20 - if (!block) { + if (block) { + /* See comment above postcopy_preempted_contains() */ + if (postcopy_preempted_contains(rs, block, offset)) { + trace_postcopy_preempt_hit(block->idstr, offset); + /* This request is dropped */ + goto again; + } + } else { /* * Poll write faults too if background snapshot is enabled; that's * when we have vcpus got blocked by the write protected pages. @@ -2173,6 +2257,114 @@ static int ram_save_target_page(RAMState *rs, PageS= earchStatus *pss, return ram_save_page(rs, pss, last_stage); } =20 +static bool postcopy_needs_preempt(RAMState *rs, PageSearchStatus *pss) +{ + /* Not enabled eager preempt? Then never do that. */ + if (!migrate_postcopy_preempt()) { + return false; + } + + /* If the ramblock we're sending is a small page? Never bother. */ + if (qemu_ram_pagesize(pss->block) =3D=3D TARGET_PAGE_SIZE) { + return false; + } + + /* Not in postcopy at all? */ + if (!migration_in_postcopy()) { + return false; + } + + /* + * If we're already handling a postcopy request, don't preempt as this= page + * has got the same high priority. + */ + if (pss->postcopy_requested) { + return false; + } + + /* If there's postcopy requests, then check it up! */ + return postcopy_has_request(rs); +} + +/* Returns true if we preempted precopy, false otherwise */ +static void postcopy_do_preempt(RAMState *rs, PageSearchStatus *pss) +{ + PostcopyPreemptState *p_state =3D &rs->postcopy_preempt_state; + + trace_postcopy_preempt_triggered(pss->block->idstr, pss->page); + + /* + * Time to preempt precopy. Cache current PSS into preempt state, so t= hat + * after handling the postcopy pages we can recover to it. We need to= do + * so because the dest VM will have partial of the precopy huge page k= ept + * over in its tmp huge page caches; better move on with it when we ca= n. + */ + p_state->ram_block =3D pss->block; + p_state->ram_page =3D pss->page; + p_state->preempted =3D true; +} + +/* Whether we're preempted by a postcopy request during sending a huge pag= e */ +static bool postcopy_preempt_triggered(RAMState *rs) +{ + return rs->postcopy_preempt_state.preempted; +} + +static void postcopy_preempt_restore(RAMState *rs, PageSearchStatus *pss) +{ + PostcopyPreemptState *state =3D &rs->postcopy_preempt_state; + + assert(state->preempted); + + pss->block =3D state->ram_block; + pss->page =3D state->ram_page; + /* This is not a postcopy request but restoring previous precopy */ + pss->postcopy_requested =3D false; + + trace_postcopy_preempt_restored(pss->block->idstr, pss->page); + + /* Reset preempt state, most importantly, set preempted=3D=3Dfalse */ + postcopy_preempt_reset(rs); +} + +static void postcopy_preempt_choose_channel(RAMState *rs, PageSearchStatus= *pss) +{ + int channel =3D pss->postcopy_requested ? RAM_CHANNEL_POSTCOPY : RAM_C= HANNEL_PRECOPY; + MigrationState *s =3D migrate_get_current(); + QEMUFile *next; + + if (channel !=3D rs->postcopy_channel) { + if (channel =3D=3D RAM_CHANNEL_PRECOPY) { + next =3D s->to_dst_file; + } else { + next =3D s->postcopy_qemufile_src; + } + /* Update and cache the current channel */ + rs->f =3D next; + rs->postcopy_channel =3D channel; + + /* + * If channel switched, reset last_sent_block since the old sent b= lock + * may not be on the same channel. + */ + rs->last_sent_block =3D NULL; + + trace_postcopy_preempt_switch_channel(channel); + } + + trace_postcopy_preempt_send_host_page(pss->block->idstr, pss->page); +} + +/* We need to make sure rs->f always points to the default channel elsewhe= re */ +static void postcopy_preempt_reset_channel(RAMState *rs) +{ + if (migrate_postcopy_preempt() && migration_in_postcopy()) { + rs->postcopy_channel =3D RAM_CHANNEL_PRECOPY; + rs->f =3D migrate_get_current()->to_dst_file; + trace_postcopy_preempt_reset_channel(); + } +} + /** * ram_save_host_page: save a whole host page * @@ -2207,7 +2399,16 @@ static int ram_save_host_page(RAMState *rs, PageSear= chStatus *pss, return 0; } =20 + if (migrate_postcopy_preempt() && migration_in_postcopy()) { + postcopy_preempt_choose_channel(rs, pss); + } + do { + if (postcopy_needs_preempt(rs, pss)) { + postcopy_do_preempt(rs, pss); + break; + } + /* Check the pages is dirty and if it is send it */ if (migration_bitmap_clear_dirty(rs, pss->block, pss->page)) { tmppages =3D ram_save_target_page(rs, pss, last_stage); @@ -2229,6 +2430,19 @@ static int ram_save_host_page(RAMState *rs, PageSear= chStatus *pss, offset_in_ramblock(pss->block, ((ram_addr_t)pss->page) << TARGET_PAGE_BIT= S)); =20 + /* + * When with postcopy preempt mode, flush the data as soon as possible= for + * postcopy requests, because we've already sent a whole huge page, so= the + * dst node should already have enough resource to atomically filling = in + * the current missing page. + * + * More importantly, when using separate postcopy channel, we must do + * explicit flush or it won't flush until the buffer is full. + */ + if (migrate_postcopy_preempt() && pss->postcopy_requested) { + qemu_fflush(rs->f); + } + res =3D ram_save_release_protection(rs, pss, start_page); return (res < 0 ? res : pages); } @@ -2272,8 +2486,17 @@ static int ram_find_and_save_block(RAMState *rs, boo= l last_stage) found =3D get_queued_page(rs, &pss); =20 if (!found) { - /* priority queue empty, so just search for something dirty */ - found =3D find_dirty_block(rs, &pss, &again); + /* + * Recover previous precopy ramblock/offset if postcopy has + * preempted precopy. Otherwise find the next dirty bit. + */ + if (postcopy_preempt_triggered(rs)) { + postcopy_preempt_restore(rs, &pss); + found =3D true; + } else { + /* priority queue empty, so just search for something dirt= y */ + found =3D find_dirty_block(rs, &pss, &again); + } } =20 if (found) { @@ -2401,6 +2624,8 @@ static void ram_state_reset(RAMState *rs) rs->last_page =3D 0; rs->last_version =3D ram_list.version; rs->xbzrle_enabled =3D false; + postcopy_preempt_reset(rs); + rs->postcopy_channel =3D RAM_CHANNEL_PRECOPY; } =20 #define MAX_WAIT 50 /* ms, half buffered_file limit */ @@ -3043,6 +3268,8 @@ static int ram_save_iterate(QEMUFile *f, void *opaque) } qemu_mutex_unlock(&rs->bitmap_mutex); =20 + postcopy_preempt_reset_channel(rs); + /* * Must occur before EOS (or any QEMUFile operation) * because of RDMA protocol. @@ -3110,6 +3337,8 @@ static int ram_save_complete(QEMUFile *f, void *opaqu= e) ram_control_after_iterate(f, RAM_CONTROL_FINISH); } =20 + postcopy_preempt_reset_channel(rs); + if (ret >=3D 0) { multifd_send_sync_main(rs->f); qemu_put_be64(f, RAM_SAVE_FLAG_EOS); @@ -3192,11 +3421,13 @@ static int load_xbzrle(QEMUFile *f, ram_addr_t addr= , void *host) * @mis: the migration incoming state pointer * @f: QEMUFile where to read the data from * @flags: Page flags (mostly to see if it's a continuation of previous bl= ock) + * @channel: the channel we're using */ static inline RAMBlock *ram_block_from_stream(MigrationIncomingState *mis, - QEMUFile *f, int flags) + QEMUFile *f, int flags, + int channel) { - RAMBlock *block =3D mis->last_recv_block; + RAMBlock *block =3D mis->last_recv_block[channel]; char id[256]; uint8_t len; =20 @@ -3223,7 +3454,7 @@ static inline RAMBlock *ram_block_from_stream(Migrati= onIncomingState *mis, return NULL; } =20 - mis->last_recv_block =3D block; + mis->last_recv_block[channel] =3D block; =20 return block; } @@ -3642,15 +3873,15 @@ int ram_postcopy_incoming_init(MigrationIncomingSta= te *mis) * rcu_read_lock is taken prior to this being called. * * @f: QEMUFile where to send the data + * @channel: the channel to use for loading */ -static int ram_load_postcopy(QEMUFile *f) +static int ram_load_postcopy(QEMUFile *f, int channel) { int flags =3D 0, ret =3D 0; bool place_needed =3D false; bool matches_target_page_size =3D false; MigrationIncomingState *mis =3D migration_incoming_get_current(); - /* Currently we only use channel 0. TODO: use all the channels */ - PostcopyTmpPage *tmp_page =3D &mis->postcopy_tmp_pages[0]; + PostcopyTmpPage *tmp_page =3D &mis->postcopy_tmp_pages[channel]; =20 while (!ret && !(flags & RAM_SAVE_FLAG_EOS)) { ram_addr_t addr; @@ -3677,7 +3908,7 @@ static int ram_load_postcopy(QEMUFile *f) trace_ram_load_postcopy_loop((uint64_t)addr, flags); if (flags & (RAM_SAVE_FLAG_ZERO | RAM_SAVE_FLAG_PAGE | RAM_SAVE_FLAG_COMPRESS_PAGE)) { - block =3D ram_block_from_stream(mis, f, flags); + block =3D ram_block_from_stream(mis, f, flags, channel); if (!block) { ret =3D -EINVAL; break; @@ -3715,10 +3946,10 @@ static int ram_load_postcopy(QEMUFile *f) } else if (tmp_page->host_addr !=3D host_page_from_ram_block_offset(block, addr)) { /* not the 1st TP within the HP */ - error_report("Non-same host page detected. Target host pa= ge %p, " - "received host page %p " + error_report("Non-same host page detected on channel %d: " + "Target host page %p, received host page %p " "(rb %s offset 0x"RAM_ADDR_FMT" target_pages = %d)", - tmp_page->host_addr, + channel, tmp_page->host_addr, host_page_from_ram_block_offset(block, addr), block->idstr, addr, tmp_page->target_pages); ret =3D -EINVAL; @@ -3818,6 +4049,28 @@ static int ram_load_postcopy(QEMUFile *f) return ret; } =20 +void *postcopy_preempt_thread(void *opaque) +{ + MigrationIncomingState *mis =3D opaque; + int ret; + + trace_postcopy_preempt_thread_entry(); + + rcu_register_thread(); + + qemu_sem_post(&mis->thread_sync_sem); + + /* Sending RAM_SAVE_FLAG_EOS to terminate this thread */ + ret =3D ram_load_postcopy(mis->postcopy_qemufile_dst, RAM_CHANNEL_POST= COPY); + + rcu_unregister_thread(); + + trace_postcopy_preempt_thread_exit(); + + return ret =3D=3D 0 ? NULL : (void *)-1; +} + + static bool postcopy_is_advised(void) { PostcopyState ps =3D postcopy_state_get(); @@ -3930,7 +4183,7 @@ static int ram_load_precopy(QEMUFile *f) =20 if (flags & (RAM_SAVE_FLAG_ZERO | RAM_SAVE_FLAG_PAGE | RAM_SAVE_FLAG_COMPRESS_PAGE | RAM_SAVE_FLAG_XBZRLE)) { - RAMBlock *block =3D ram_block_from_stream(mis, f, flags); + RAMBlock *block =3D ram_block_from_stream(mis, f, flags, RAM_C= HANNEL_PRECOPY); =20 host =3D host_from_ram_block_offset(block, addr); /* @@ -4107,7 +4360,12 @@ static int ram_load(QEMUFile *f, void *opaque, int v= ersion_id) */ WITH_RCU_READ_LOCK_GUARD() { if (postcopy_running) { - ret =3D ram_load_postcopy(f); + /* + * Note! Here RAM_CHANNEL_PRECOPY is the precopy channel of + * postcopy migration, we have another RAM_CHANNEL_POSTCOPY to + * service fast page faults. + */ + ret =3D ram_load_postcopy(f, RAM_CHANNEL_PRECOPY); } else { ret =3D ram_load_precopy(f); } @@ -4269,6 +4527,12 @@ static int ram_resume_prepare(MigrationState *s, voi= d *opaque) return 0; } =20 +void postcopy_preempt_shutdown_file(MigrationState *s) +{ + qemu_put_be64(s->postcopy_qemufile_src, RAM_SAVE_FLAG_EOS); + qemu_fflush(s->postcopy_qemufile_src); +} + static SaveVMHandlers savevm_ram_handlers =3D { .save_setup =3D ram_save_setup, .save_live_iterate =3D ram_save_iterate, diff --git a/migration/ram.h b/migration/ram.h index 2c6dc3675d..f31b8c0ece 100644 --- a/migration/ram.h +++ b/migration/ram.h @@ -72,6 +72,8 @@ int64_t ramblock_recv_bitmap_send(QEMUFile *file, const char *block_name); int ram_dirty_bitmap_reload(MigrationState *s, RAMBlock *rb); bool ramblock_page_is_discarded(RAMBlock *rb, ram_addr_t start); +void postcopy_preempt_shutdown_file(MigrationState *s); +void *postcopy_preempt_thread(void *opaque); =20 /* ram cache */ int colo_init_ram_cache(void); diff --git a/migration/socket.c b/migration/socket.c index 05705a32d8..955c5ebb10 100644 --- a/migration/socket.c +++ b/migration/socket.c @@ -39,6 +39,24 @@ void socket_send_channel_create(QIOTaskFunc f, void *dat= a) f, data, NULL, NULL); } =20 +QIOChannel *socket_send_channel_create_sync(Error **errp) +{ + QIOChannelSocket *sioc =3D qio_channel_socket_new(); + + if (!outgoing_args.saddr) { + object_unref(OBJECT(sioc)); + error_setg(errp, "Initial sock address not set!"); + return NULL; + } + + if (qio_channel_socket_connect_sync(sioc, outgoing_args.saddr, errp) <= 0) { + object_unref(OBJECT(sioc)); + return NULL; + } + + return QIO_CHANNEL(sioc); +} + int socket_send_channel_destroy(QIOChannel *send) { /* Remove channel */ diff --git a/migration/socket.h b/migration/socket.h index 891dbccceb..dc54df4e6c 100644 --- a/migration/socket.h +++ b/migration/socket.h @@ -21,6 +21,7 @@ #include "io/task.h" =20 void socket_send_channel_create(QIOTaskFunc f, void *data); +QIOChannel *socket_send_channel_create_sync(Error **errp); int socket_send_channel_destroy(QIOChannel *send); =20 void socket_start_incoming_migration(const char *str, Error **errp); diff --git a/migration/trace-events b/migration/trace-events index 3a9b3567ae..6452179bee 100644 --- a/migration/trace-events +++ b/migration/trace-events @@ -110,6 +110,12 @@ ram_save_iterate_big_wait(uint64_t milliconds, int ite= rations) "big wait: %" PRI ram_load_complete(int ret, uint64_t seq_iter) "exit_code %d seq iteration = %" PRIu64 ram_write_tracking_ramblock_start(const char *block_id, size_t page_size, = void *addr, size_t length) "%s: page_size: %zu addr: %p length: %zu" ram_write_tracking_ramblock_stop(const char *block_id, size_t page_size, v= oid *addr, size_t length) "%s: page_size: %zu addr: %p length: %zu" +postcopy_preempt_triggered(char *str, unsigned long page) "during sending = ramblock %s offset 0x%lx" +postcopy_preempt_restored(char *str, unsigned long page) "ramblock %s offs= et 0x%lx" +postcopy_preempt_hit(char *str, uint64_t offset) "ramblock %s offset 0x%"P= RIx64 +postcopy_preempt_send_host_page(char *str, uint64_t offset) "ramblock %s o= ffset 0x%"PRIx64 +postcopy_preempt_switch_channel(int channel) "%d" +postcopy_preempt_reset_channel(void) "" =20 # multifd.c multifd_new_send_channel_async(uint8_t id) "channel %d" @@ -175,6 +181,7 @@ migration_thread_low_pending(uint64_t pending) "%" PRIu= 64 migrate_transferred(uint64_t tranferred, uint64_t time_spent, uint64_t ban= dwidth, uint64_t size) "transferred %" PRIu64 " time_spent %" PRIu64 " band= width %" PRIu64 " max_size %" PRId64 process_incoming_migration_co_end(int ret, int ps) "ret=3D%d postcopy-stat= e=3D%d" process_incoming_migration_co_postcopy_end_main(void) "" +postcopy_preempt_enabled(bool value) "%d" =20 # channel.c migration_set_incoming_channel(void *ioc, const char *ioctype) "ioc=3D%p i= octype=3D%s" @@ -277,6 +284,9 @@ postcopy_request_shared_page(const char *sharer, const = char *rb, uint64_t rb_off postcopy_request_shared_page_present(const char *sharer, const char *rb, u= int64_t rb_offset) "%s already %s offset 0x%"PRIx64 postcopy_wake_shared(uint64_t client_addr, const char *rb) "at 0x%"PRIx64"= in %s" postcopy_page_req_del(void *addr, int count) "resolved page req %p total %= d" +postcopy_preempt_new_channel(void) "" +postcopy_preempt_thread_entry(void) "" +postcopy_preempt_thread_exit(void) "" =20 get_mem_fault_cpu_index(int cpu, uint32_t pid) "cpu: %d, pid: %u" =20 --=20 2.32.0 From nobody Sat Apr 27 09:25:08 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1642581286894151.04052914961528; Wed, 19 Jan 2022 00:34:46 -0800 (PST) Received: from localhost ([::1]:52270 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nA6QX-0006B2-SY for importer@patchew.org; Wed, 19 Jan 2022 03:34:45 -0500 Received: from eggs.gnu.org ([209.51.188.92]:59142) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nA63Q-0007Wd-8C for qemu-devel@nongnu.org; Wed, 19 Jan 2022 03:10:52 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:20514) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nA63N-0001Pk-8t for qemu-devel@nongnu.org; Wed, 19 Jan 2022 03:10:51 -0500 Received: from mail-wm1-f72.google.com (mail-wm1-f72.google.com [209.85.128.72]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-540-Eawx3yc3Ngih6CUxBsUTRg-1; Wed, 19 Jan 2022 03:10:43 -0500 Received: by mail-wm1-f72.google.com with SMTP id n14-20020a7bcbce000000b003488820f0d9so1323544wmi.8 for ; Wed, 19 Jan 2022 00:10:43 -0800 (PST) Received: from localhost.localdomain ([85.203.46.175]) by smtp.gmail.com with ESMTPSA id a20sm4351606wmb.27.2022.01.19.00.10.38 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Wed, 19 Jan 2022 00:10:41 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1642579845; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=DMzUc77KxXCbJobFky9F7XYbxJfA3c/ZqHxZ4FS5G0E=; b=J4keMbvxsfRMlKGdEZjfrEpumomBhy2KbAkXdmBdwBc0MWJUxrfvdIRAyD3LMKBzC3/Qfn GKtwMUqFgjXxmznH62hJm60hIPyKclFRWozCss6ybX5s4aLr+DyWlGrBxXKmkhGNnADuIZ VMvuPggQwdu1Ecx3OW0Y6fOR5bXCzUU= X-MC-Unique: Eawx3yc3Ngih6CUxBsUTRg-1 X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=DMzUc77KxXCbJobFky9F7XYbxJfA3c/ZqHxZ4FS5G0E=; b=pbii5znAL77IU2pkMRCBYdSAPt5d92D9MBfZEltAHxMOl4kuYIoyAAswTRX9dbP0cP SvyGbPaSiz01jMEIAxKSyo0V8tR2qoLr10PPDR6gv3UceivTTawosBZ61KhidBiXEBQY M0AR39pgJAnbVDqxCkLzWvZePe7rN1hoQThXxyRyJf3k4FBYOpLj1PgC6GQIVahsgfYo EjbgG3IVwqriMmB7J35eLIdnsCZ7dUzxT9yMfTdoz5T5bic3ylRsz143GbN59/2Gocej WqZO+10mmQ8itXHcLufG+qsbnn+EAfXpkz+GY4STinlLQ7sfCtSk/G5aJIsCTZPKodem tTEg== X-Gm-Message-State: AOAM532QjRMWQsRH2ufYWOIoZZzP5GOwxehuNFxothRN/zI8+O+cJ5e5 45u/mf8jr8p4iAWEht7Ukpt+VYWH88NlqS9QA6W1Ih2Rt/jV8PTcrS6s/eRStETMQZ+fylU9rgf r4WBaNoQwsoTCUFb8LZD0nLE0y7wPxYOjFIZiddnFUEeDELFOzzdhUtaELMVXjc74 X-Received: by 2002:a05:6000:2ac:: with SMTP id l12mr17836084wry.165.1642579842256; Wed, 19 Jan 2022 00:10:42 -0800 (PST) X-Google-Smtp-Source: ABdhPJys4Yr09AYXaBCEOZ4tRV8Vi8PQP8EDS+OVXrtI8++8Pf5kj5RimFTAV37Vy/qSyWs04pJTOA== X-Received: by 2002:a05:6000:2ac:: with SMTP id l12mr17836064wry.165.1642579842061; Wed, 19 Jan 2022 00:10:42 -0800 (PST) From: Peter Xu To: qemu-devel@nongnu.org Subject: [PATCH RFC 15/15] tests: Add postcopy preempt test Date: Wed, 19 Jan 2022 16:09:29 +0800 Message-Id: <20220119080929.39485-16-peterx@redhat.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20220119080929.39485-1-peterx@redhat.com> References: <20220119080929.39485-1-peterx@redhat.com> MIME-Version: 1.0 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=peterx@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=peterx@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -34 X-Spam_score: -3.5 X-Spam_bar: --- X-Spam_report: (-3.5 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.7, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Juan Quintela , "Dr . David Alan Gilbert" , peterx@redhat.com, Leonardo Bras Soares Passos Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZM-MESSAGEID: 1642581287995100001 Content-Type: text/plain; charset="utf-8" Signed-off-by: Peter Xu Reviewed-by: Dr. David Alan Gilbert --- tests/qtest/migration-test.c | 21 +++++++++++++++++++++ 1 file changed, 21 insertions(+) diff --git a/tests/qtest/migration-test.c b/tests/qtest/migration-test.c index 7b42f6fd90..93ff43bb3f 100644 --- a/tests/qtest/migration-test.c +++ b/tests/qtest/migration-test.c @@ -470,6 +470,7 @@ typedef struct { */ bool hide_stderr; bool use_shmem; + bool postcopy_preempt; /* only launch the target process */ bool only_target; /* Use dirty ring if true; dirty logging otherwise */ @@ -673,6 +674,11 @@ static int migrate_postcopy_prepare(QTestState **from_= ptr, migrate_set_capability(to, "postcopy-ram", true); migrate_set_capability(to, "postcopy-blocktime", true); =20 + if (args->postcopy_preempt) { + migrate_set_capability(from, "postcopy-preempt", true); + migrate_set_capability(to, "postcopy-preempt", true); + } + /* We want to pick a speed slow enough that the test completes * quickly, but that it doesn't complete precopy even on a slow * machine, so also set the downtime. @@ -719,6 +725,20 @@ static void test_postcopy(void) migrate_postcopy_complete(from, to); } =20 +static void test_postcopy_preempt(void) +{ + MigrateStart *args =3D migrate_start_new(); + QTestState *from, *to; + + args->postcopy_preempt =3D true; + + if (migrate_postcopy_prepare(&from, &to, args)) { + return; + } + migrate_postcopy_start(from, to); + migrate_postcopy_complete(from, to); +} + static void test_postcopy_recovery(void) { MigrateStart *args =3D migrate_start_new(); @@ -1458,6 +1478,7 @@ int main(int argc, char **argv) module_call_init(MODULE_INIT_QOM); =20 qtest_add_func("/migration/postcopy/unix", test_postcopy); + qtest_add_func("/migration/postcopy/preempt", test_postcopy_preempt); qtest_add_func("/migration/postcopy/recovery", test_postcopy_recovery); qtest_add_func("/migration/bad_dest", test_baddest); qtest_add_func("/migration/precopy/unix", test_precopy_unix); --=20 2.32.0