From nobody Fri Apr 26 23:13:09 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1663719084; cv=none; d=zohomail.com; s=zohoarc; b=isfbu1rA3ZNTojSHkJZ8LHda7nGl/JQPjdqhp1xK5yhG2xdR+Azv9ztKc5nDIWE2d65n823WkjV7tBMkC3PJEDzv9+y/UY2LdhAMphMVtXsgvaSlcgIPFPjpRaIWVkE/af8oPPhSZCqeCVD7Ye4NXH5RkWESxpBHR3tqhWhyR1o= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1663719084; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=nrFe7joiBGxWYq/BylgVOGeA8n5rVAbNSYrRgCoSmoE=; b=HbfIkop9pdKJ36rR3s5AIA5pLAWM/GbzLA65Brm+t7cE3DR9hyTOn37hzTElASJF+RDI6z1SsI58eMrP50FVNKlKNB/yOK/Ww1hReH/FIwpzMYGf9tJmgYcw+jYUv4Bi6OMxtUyUvCSG9sRkSY13cqHhnypuAFzOd5nGzsyR8ec= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1663719084895157.7778440911527; Tue, 20 Sep 2022 17:11:24 -0700 (PDT) Received: from localhost ([::1]:39258 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1oanKl-0002mE-Cg for importer@patchew.org; Tue, 20 Sep 2022 20:11:23 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:44072) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1oam5D-0003lA-Gu for qemu-devel@nongnu.org; Tue, 20 Sep 2022 18:51:15 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]:59279) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1oam5A-0001QF-RA for qemu-devel@nongnu.org; Tue, 20 Sep 2022 18:51:15 -0400 Received: from mail-qv1-f69.google.com (mail-qv1-f69.google.com [209.85.219.69]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_128_GCM_SHA256) id us-mta-265-ywqhi4rxNgmHCiFACnqFWg-1; Tue, 20 Sep 2022 18:51:11 -0400 Received: by mail-qv1-f69.google.com with SMTP id c1-20020a0cfb01000000b00495ad218c74so3065444qvp.20 for ; Tue, 20 Sep 2022 15:51:10 -0700 (PDT) Received: from localhost.localdomain (bras-base-aurron9127w-grc-46-70-31-27-79.dsl.bell.ca. [70.31.27.79]) by smtp.gmail.com with ESMTPSA id j12-20020ac8440c000000b0035d0655b079sm275470qtn.30.2022.09.20.15.51.08 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Tue, 20 Sep 2022 15:51:09 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1663714272; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=nrFe7joiBGxWYq/BylgVOGeA8n5rVAbNSYrRgCoSmoE=; b=HQsJ+2hSocNgkYEZ9JDTJnqYR6REqN5l5vsX3MBwl8G6Vj47D1EiJ/4x494BCT0e8tokyr sM5+/ZkgpAreVWjKg32D9K2gz8CxdQTZC0o9GvQoffNLCsE0EWpEvIp3ac5+1wOeshidKx r6O/4ImLGsJSYJtYEf45XIlcIHaJeMc= X-MC-Unique: ywqhi4rxNgmHCiFACnqFWg-1 X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date; bh=nrFe7joiBGxWYq/BylgVOGeA8n5rVAbNSYrRgCoSmoE=; b=hSHkcskBmv3m0/gL66A3HPu3Gz7qH/5OD9am+h0SW8zSagwro7NisBxhn0Ba9zQUzG wMQhbkNqkQxrzqEzqfgAESIN2M2lVtqfVK2EFbPLw9fLBgxcN1pJZexD2blbyG9McyCP mwhQuqfvftk87hkSmhnr4d3R4KBnJX+F15ffvwUMnH+ArigBkYZWQvFapuomnyc0t5vt GLfitNsmM2DY16fnuNaJJAqat2IMneBzYIB7Za9K4yHiEDWsVO8U2JdUCe7fggagwPI5 mQz3qIroxa0QRMdsjZKt6sTjPV70ovBTn9xeGtdkeMZ29Lcx4rJf+vLdCYLQaee2sqjZ 9PFQ== X-Gm-Message-State: ACrzQf0b8rHuY5kykKRLrzorzIyxRqMnaYKx/LhKE8OJVubW2jnKK0Jd mlv4OxxV6GhK1XP1ftbOpUnMbeuGDcP3wjjssq5sQo1Xw+sLvVyANfH7LLGJDTvhNakeyoCYh3m xi3qC2gqrT3zmkrL7fTIxhf2k4eZyIa8TP3qyHoxxvISWMtXRReKWJvgUShPDh1xX X-Received: by 2002:a05:620a:490a:b0:6ce:d88f:30fe with SMTP id ed10-20020a05620a490a00b006ced88f30femr13484513qkb.534.1663714270157; Tue, 20 Sep 2022 15:51:10 -0700 (PDT) X-Google-Smtp-Source: AMsMyM5R/fIYX222ygjCJQ8pnogOuVhm+XeuKINlmp67r7LfNq5bbx440igQazEksRzjHVXuLCh4XQ== X-Received: by 2002:a05:620a:490a:b0:6ce:d88f:30fe with SMTP id ed10-20020a05620a490a00b006ced88f30femr13484497qkb.534.1663714269900; Tue, 20 Sep 2022 15:51:09 -0700 (PDT) From: Peter Xu To: qemu-devel@nongnu.org Cc: "Dr . David Alan Gilbert" , Manish Mishra , peterx@redhat.com, Juan Quintela , Leonardo Bras Soares Passos , ani@anisinha.ca, "Daniel P . Berrange" Subject: [PATCH 01/14] migration: Add postcopy_preempt_active() Date: Tue, 20 Sep 2022 18:50:53 -0400 Message-Id: <20220920225106.48451-2-peterx@redhat.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20220920225106.48451-1-peterx@redhat.com> References: <20220920225106.48451-1-peterx@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=peterx@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1663719086630100001 Content-Type: text/plain; charset="utf-8" Add the helper to show that postcopy preempt enabled, meanwhile active. Reviewed-by: Dr. David Alan Gilbert Signed-off-by: Peter Xu --- migration/ram.c | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-) diff --git a/migration/ram.c b/migration/ram.c index 1d42414ecc..d8cf7cc901 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -162,6 +162,11 @@ out: return ret; } =20 +static bool postcopy_preempt_active(void) +{ + return migrate_postcopy_preempt() && migration_in_postcopy(); +} + bool ramblock_is_ignored(RAMBlock *block) { return !qemu_ram_is_migratable(block) || @@ -2434,7 +2439,7 @@ static void postcopy_preempt_choose_channel(RAMState = *rs, PageSearchStatus *pss) /* We need to make sure rs->f always points to the default channel elsewhe= re */ static void postcopy_preempt_reset_channel(RAMState *rs) { - if (migrate_postcopy_preempt() && migration_in_postcopy()) { + if (postcopy_preempt_active()) { rs->postcopy_channel =3D RAM_CHANNEL_PRECOPY; rs->f =3D migrate_get_current()->to_dst_file; trace_postcopy_preempt_reset_channel(); @@ -2472,7 +2477,7 @@ static int ram_save_host_page(RAMState *rs, PageSearc= hStatus *pss) return 0; } =20 - if (migrate_postcopy_preempt() && migration_in_postcopy()) { + if (postcopy_preempt_active()) { postcopy_preempt_choose_channel(rs, pss); } =20 --=20 2.32.0 From nobody Fri Apr 26 23:13:09 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1663717810; cv=none; d=zohomail.com; s=zohoarc; b=NUgBfw741sNQcmX+S1hinsX2jsG6rHDZsopyhFxI8psEdwUYn2dQPvQRWoOIag25th9ZPFQQUVjZ2f2gsQbQL89jqY0Pih4b5El6FGvznmhz+TaF9b7TPOCbKftseP09q+aOJtk38mv033cjItkZ6aIo+DcRup1IcG0FTclGrW8= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1663717810; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=fBntkORANdSFcod4owFYGT32WIlQPkv35mQCzO/lmko=; b=dcYoN3mZCYALzmxS/kB10M6gc1SDG00YUFSzE6OFLkq5Fu08oWSJny42yvCW6ZnbxOxl0QYB4L8tAzGhHB4w2KMwsXbSXq7toOF4naUwYWhIV0vrPNIyKNjTXsSRWbI06KbDaXird1s6UJtpnB3cCwxSTANNDu4MG8Z3ITYxdiA= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 16637178106271015.1567809892507; Tue, 20 Sep 2022 16:50:10 -0700 (PDT) Received: from localhost ([::1]:52704 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1oan0D-0005QC-Eh for importer@patchew.org; Tue, 20 Sep 2022 19:50:09 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:55932) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1oam5D-0003lO-K9 for qemu-devel@nongnu.org; Tue, 20 Sep 2022 18:51:15 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:38753) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1oam5C-0001QQ-0P for qemu-devel@nongnu.org; Tue, 20 Sep 2022 18:51:15 -0400 Received: from mail-qt1-f197.google.com (mail-qt1-f197.google.com [209.85.160.197]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_128_GCM_SHA256) id us-mta-632-cQU4iLcHNZSYrHQFpLXbdA-1; Tue, 20 Sep 2022 18:51:12 -0400 Received: by mail-qt1-f197.google.com with SMTP id e22-20020ac84b56000000b0035bb64ad562so2898110qts.17 for ; Tue, 20 Sep 2022 15:51:12 -0700 (PDT) Received: from localhost.localdomain (bras-base-aurron9127w-grc-46-70-31-27-79.dsl.bell.ca. [70.31.27.79]) by smtp.gmail.com with ESMTPSA id j12-20020ac8440c000000b0035d0655b079sm275470qtn.30.2022.09.20.15.51.10 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Tue, 20 Sep 2022 15:51:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1663714273; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=fBntkORANdSFcod4owFYGT32WIlQPkv35mQCzO/lmko=; b=WQ8hR1G1yN2KnkDOZv8DW+N7NA05zdfCWtd9+6XY5RFaBFrK9eDan7vsHTvi9qQn/v8dEG lb7T6o1A6espThe/z1CIvOa+wYMJ6bCG40gNihNceWam5F6zfNSR/jXDGYMkpmq8Y8x4MQ FO9QL5ZNs903F3dfoP6reniB46l32jI= X-MC-Unique: cQU4iLcHNZSYrHQFpLXbdA-1 X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date; bh=fBntkORANdSFcod4owFYGT32WIlQPkv35mQCzO/lmko=; b=wg8u0K8PS4ACWKTSy4KUKD28g8onqvWYti4L+Kj4IAaZvb7aeA/LjcUObljUJJZv8z cso/17BfqiECufPXSbdpyN26rol6DgGdqdRWHkTnNBLkPfyAO16k+d5aHcmMDSThJndV G8MIWDD/mI6gF0UAklrDMTgNjQcgz8ZQMkG3QceyOkeFYisB/70rBKQOtSjXowY6Ir2C U39OKvCsLkEuNGu+RiXrZvVSI2ELuDiyGFxtPDwC4Jia15wY6hd5iioWvF0zDCEToyGl T2iuXGytGf4nN47jG9mouwkMNAAK/gdOvrAmLPYz9ast31FjOdlioDZPRiNLDYFbIww7 nJbg== X-Gm-Message-State: ACrzQf1JjIOuSingZ6GPB7beLGL0UmSsKGL4FkebRR4HMf4ax8rTedzS HUSQ6eOvD2k/vwmzVl/l9VyO/i6FlYkUuZCPvO9mEzRh0VgJvUPSRQNktp8JrP1NFK0g3xKK2rJ PRfXgEeUghrl7cSkS3ghPoLLNnQjG5BDr2f4yeAq2rZpR9g86Rk+GrEWW9wIIV989 X-Received: by 2002:a05:6214:242c:b0:4a7:d1cd:427c with SMTP id gy12-20020a056214242c00b004a7d1cd427cmr21098951qvb.45.1663714271657; Tue, 20 Sep 2022 15:51:11 -0700 (PDT) X-Google-Smtp-Source: AMsMyM6vLvSwHSxZ2fg3J1Qk5vXLk0KdN1HZFHUdlbX5LWcD6oiFgSELPGNYprJGgW3it7qhTeJt9g== X-Received: by 2002:a05:6214:242c:b0:4a7:d1cd:427c with SMTP id gy12-20020a056214242c00b004a7d1cd427cmr21098933qvb.45.1663714271347; Tue, 20 Sep 2022 15:51:11 -0700 (PDT) From: Peter Xu To: qemu-devel@nongnu.org Cc: "Dr . David Alan Gilbert" , Manish Mishra , peterx@redhat.com, Juan Quintela , Leonardo Bras Soares Passos , ani@anisinha.ca, "Daniel P . Berrange" Subject: [PATCH 02/14] migration: Cleanup xbzrle zero page cache update logic Date: Tue, 20 Sep 2022 18:50:54 -0400 Message-Id: <20220920225106.48451-3-peterx@redhat.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20220920225106.48451-1-peterx@redhat.com> References: <20220920225106.48451-1-peterx@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=peterx@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1663717811512100001 Content-Type: text/plain; charset="utf-8" The major change is to replace "!save_page_use_compression()" with "xbzrle_enabled" to make it clear. Reasonings: (1) When compression enabled, "!save_page_use_compression()" is exactly the same as checking "xbzrle_enabled". (2) When compression disabled, "!save_page_use_compression()" always return true. We used to try calling the xbzrle code, but after this change we won't, and we shouldn't need to. Since at it, drop the xbzrle_enabled check in xbzrle_cache_zero_page() because with this change it's not needed anymore. Signed-off-by: Peter Xu Reviewed-by: Dr. David Alan Gilbert --- migration/ram.c | 6 +----- 1 file changed, 1 insertion(+), 5 deletions(-) diff --git a/migration/ram.c b/migration/ram.c index d8cf7cc901..fc59c052cf 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -741,10 +741,6 @@ void mig_throttle_counter_reset(void) */ static void xbzrle_cache_zero_page(RAMState *rs, ram_addr_t current_addr) { - if (!rs->xbzrle_enabled) { - return; - } - /* We don't care if this fails to allocate a new cache page * as long as it updated an old one */ cache_insert(XBZRLE.cache, current_addr, XBZRLE.zero_target_page, @@ -2301,7 +2297,7 @@ static int ram_save_target_page(RAMState *rs, PageSea= rchStatus *pss) /* Must let xbzrle know, otherwise a previous (now 0'd) cached * page would be stale */ - if (!save_page_use_compression(rs)) { + if (rs->xbzrle_enabled) { XBZRLE_cache_lock(); xbzrle_cache_zero_page(rs, block->offset + offset); XBZRLE_cache_unlock(); --=20 2.32.0 From nobody Fri Apr 26 23:13:09 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1663717995; cv=none; d=zohomail.com; s=zohoarc; b=nqhbh/u04vzDKNGruHohQ6wpwhyvHloPaMZZIOqc1uJW1LDk4SgE1iJm/9/RoITck1jFYZhQ80T59rD+OyOiYUG1m22aNFFE+3sx117iP+vzmdkGtEjLSZfbBK9+dDHpGWitVsCdzS7ef7DZ0MdBTBr6g18Im9myDydLWLrHt/U= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1663717995; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=qJN8+CcKlnYyD18fUmuo4UQ+WkRykLBYx3hA/wBQj6o=; b=ZowxNa8iy8puMqUqvDSOURfklAJVW6j6Hf2EePpeAi0UDKBszaZLAIVm60gjKf/crPlG52IELjF/PVtzBt0LbGspKusgZffI6yF0aJRsKukRPYdcdegxkGzmyp+mrj4pkK/g42rCK4E9S51/pQF8hnE0ehP0dZdeKHZySajjkuo= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1663717995341528.9312226652146; Tue, 20 Sep 2022 16:53:15 -0700 (PDT) Received: from localhost ([::1]:49738 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1oan3C-0003et-29 for importer@patchew.org; Tue, 20 Sep 2022 19:53:14 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:55934) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1oam5E-0003oc-Kh for qemu-devel@nongnu.org; Tue, 20 Sep 2022 18:51:16 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]:40294) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1oam5D-0001Qn-5o for qemu-devel@nongnu.org; Tue, 20 Sep 2022 18:51:16 -0400 Received: from mail-qt1-f200.google.com (mail-qt1-f200.google.com [209.85.160.200]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_128_GCM_SHA256) id us-mta-657--I12DEW8MJWuVXHl-I1T0g-1; Tue, 20 Sep 2022 18:51:13 -0400 Received: by mail-qt1-f200.google.com with SMTP id ew15-20020a05622a514f00b0035cb76e35e5so2877696qtb.7 for ; Tue, 20 Sep 2022 15:51:13 -0700 (PDT) Received: from localhost.localdomain (bras-base-aurron9127w-grc-46-70-31-27-79.dsl.bell.ca. [70.31.27.79]) by smtp.gmail.com with ESMTPSA id j12-20020ac8440c000000b0035d0655b079sm275470qtn.30.2022.09.20.15.51.11 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Tue, 20 Sep 2022 15:51:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1663714274; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=qJN8+CcKlnYyD18fUmuo4UQ+WkRykLBYx3hA/wBQj6o=; b=HkjL4md2CvMiXi6zrmEidP8CLX8+vL9rxYXyOjjYk7u/R+4R084HRDZeXRSWWDCmctmUKx OeB6C/4+966KZ6bUwrftizDHyzVVJ7Bw+PHHwlACAEBXjNlfk0Ne9zGnJyuFDdqis3a3tl bLzZrvMb+KzO0ZabOBtusmq5wamMcJo= X-MC-Unique: -I12DEW8MJWuVXHl-I1T0g-1 X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date; bh=qJN8+CcKlnYyD18fUmuo4UQ+WkRykLBYx3hA/wBQj6o=; b=5F1RbhUsoBKzzV3evlRM6gpUkSHghVLEmsCywzDX3t3TwszTswj/Qd702LY9UgFNxW kWKiDe8yK1KhEvMJQpHURrHbPF1SEj9lQfNd48DarR1QRxkmWVmBSmcleeZkeZr5VzIW J3096yS/ey8oy/6EVLlLoJ2dkaGn2M4CRiIgWXfc/0ZE2DbwfnVjTMN4lTIw5tSFjBlS 9NXQo/QbV0w/Dt9xREhNo2+Zqil4SZNtFf7aSYRwgL+gxcW9jnnk9T04PvVKmW570vnB IiyG4DRbqhmmPGPa7qmqdg3M6/fAYGyb+kY852Xj02Rswav6pQLpDaMpIpdBlVhGIrCx cC+A== X-Gm-Message-State: ACrzQf06qpUXTut5VW/kvbF3gX2TARQjEL8pcI5WZmLvg9/6ceR75Hug lcBp1tBBoxz4fw62bON95Xr+Tnp0Tk58COPIf50BYSqyt4/D1yHWcUlYLexuthCghAHNIV1+Axp dKG0GoTv95V86j2q6Uj9bIcETaA88XdvSOKwk/uZ/gRmGTc8IpUEKvCHZMxfIr6w7 X-Received: by 2002:a05:6214:1d2e:b0:4ad:df6:96e5 with SMTP id f14-20020a0562141d2e00b004ad0df696e5mr20307948qvd.100.1663714272889; Tue, 20 Sep 2022 15:51:12 -0700 (PDT) X-Google-Smtp-Source: AMsMyM6vvCG3bJRFR64pKItfApziN9fhgqI30bCH/Yo98MT/HvnvaFLEttS0OS6Vh2ucGB9FeLgpWQ== X-Received: by 2002:a05:6214:1d2e:b0:4ad:df6:96e5 with SMTP id f14-20020a0562141d2e00b004ad0df696e5mr20307928qvd.100.1663714272680; Tue, 20 Sep 2022 15:51:12 -0700 (PDT) From: Peter Xu To: qemu-devel@nongnu.org Cc: "Dr . David Alan Gilbert" , Manish Mishra , peterx@redhat.com, Juan Quintela , Leonardo Bras Soares Passos , ani@anisinha.ca, "Daniel P . Berrange" Subject: [PATCH 03/14] migration: Trivial cleanup save_page_header() on same block check Date: Tue, 20 Sep 2022 18:50:55 -0400 Message-Id: <20220920225106.48451-4-peterx@redhat.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20220920225106.48451-1-peterx@redhat.com> References: <20220920225106.48451-1-peterx@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=peterx@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1663717996767100001 Content-Type: text/plain; charset="utf-8" The 2nd check on RAM_SAVE_FLAG_CONTINUE is a bit redundant. Use a boolean to be clearer. Signed-off-by: Peter Xu Reviewed-by: Dr. David Alan Gilbert --- migration/ram.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/migration/ram.c b/migration/ram.c index fc59c052cf..62ff2c1469 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -661,14 +661,15 @@ static size_t save_page_header(RAMState *rs, QEMUFile= *f, RAMBlock *block, ram_addr_t offset) { size_t size, len; + bool same_block =3D (block =3D=3D rs->last_sent_block); =20 - if (block =3D=3D rs->last_sent_block) { + if (same_block) { offset |=3D RAM_SAVE_FLAG_CONTINUE; } qemu_put_be64(f, offset); size =3D 8; =20 - if (!(offset & RAM_SAVE_FLAG_CONTINUE)) { + if (!same_block) { len =3D strlen(block->idstr); qemu_put_byte(f, len); qemu_put_buffer(f, (uint8_t *)block->idstr, len); --=20 2.32.0 From nobody Fri Apr 26 23:13:09 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1663721687; cv=none; d=zohomail.com; s=zohoarc; b=TiBmSk1kRsSpIIyygS95Lrt49BqAtZu1v/leihU4gfdb28YaKOXbI8smuENOFndH2PVGrnKfJuNrt5ohtcno7H8C92YIY45pji9vfKQk4/YXZg1ofK2oFN/qRe7OMJ4ubQwGx/Ivdto24/0C1lFSkk5E1YPAsRDhWAsDc8MvCRE= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1663721687; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=MpBYvfeGYvDWgqyMt+UF+aJyk6vZBNjle9m84bxLNX4=; b=l6O9uf+V4lbxfnCQyF/2eleCVG2eXQ19AW6LjAOHtAtiyArYhOsFFydDOVZ/RMQX2c0QPXrorIvPBTjUBIfsXxIPCoAqOT7XmQ4rIjiYkvOlwlLRAUxJ1vgD0jbolcmvdAiAye0vUyzupWFTZHibHCQTKGbVhOc1SS98vetqZmo= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1663721687389183.4496181469642; Tue, 20 Sep 2022 17:54:47 -0700 (PDT) Received: from localhost ([::1]:35312 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1oao0k-0003E7-2d for importer@patchew.org; Tue, 20 Sep 2022 20:54:46 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:55936) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1oam5G-0003tf-Ln for qemu-devel@nongnu.org; Tue, 20 Sep 2022 18:51:18 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:41689) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1oam5F-0001RJ-20 for qemu-devel@nongnu.org; Tue, 20 Sep 2022 18:51:18 -0400 Received: from mail-qv1-f70.google.com (mail-qv1-f70.google.com [209.85.219.70]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_128_GCM_SHA256) id us-mta-646-FqbBpovCM2SK9_b_58lqWw-1; Tue, 20 Sep 2022 18:51:15 -0400 Received: by mail-qv1-f70.google.com with SMTP id m7-20020a0ce6e7000000b004ad69308f01so902540qvn.9 for ; Tue, 20 Sep 2022 15:51:15 -0700 (PDT) Received: from localhost.localdomain (bras-base-aurron9127w-grc-46-70-31-27-79.dsl.bell.ca. [70.31.27.79]) by smtp.gmail.com with ESMTPSA id j12-20020ac8440c000000b0035d0655b079sm275470qtn.30.2022.09.20.15.51.12 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Tue, 20 Sep 2022 15:51:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1663714276; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=MpBYvfeGYvDWgqyMt+UF+aJyk6vZBNjle9m84bxLNX4=; b=DnLr3/nFOviYaXXUZcP+4xXIri8vKS9NMNpLsbZ2LTGFGNQb1f2yEG5bPtdLjdmVPqwKhD KZP5+6ls3KTlk73QFuUhfSi3wikSAGx7+p71KXPmyCVRd4PNPzebxdsOFDR2ibPTbA712z o3L0OzVgAq/+neebsP52gwc9D1cSFaw= X-MC-Unique: FqbBpovCM2SK9_b_58lqWw-1 X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date; bh=MpBYvfeGYvDWgqyMt+UF+aJyk6vZBNjle9m84bxLNX4=; b=hoybOAJ8JWXp736rTuOjK1I6q8G6U99ntiNPeGC2Q/rfwHRVNCgZOqSMHNGxNO4K9z gKeRKo9ev2e3+w0cZg8gGyXZfLphUN7A/9NglsYxlBB8b6bV31ryrz0uruLnRtaN+nEd E2NGHkHen7zpw1IXZ5RPVpUzLrOkJCRigwptXLeMEmH4NpH0vL6DxBX5uAKW8XIUNU/5 Yy9Qv1i0VznbIb35+Fqpf3SwNZfIbhwRPHI8VSvSqJgOzG/iroKnNe7opzQyvM/ZCbgp 1ldFDqBRZA5ISXxqnsD0HUTrl6xfnml+wYVE2rvZRtj0F0zM2ZQzpdH6DeZqD6z3ff7Q 2IQw== X-Gm-Message-State: ACrzQf0JzJcpACDhVL8VQmB2ZR0AtecrPDH88A3Pusgzn9j7HpYX4obx 7YOV4Q2kaOlfx8AWj+b3KRgNnnvzDMpg3Dg7YUWg4r8hkmM6Z9LwsNgXilOy+lJClfTBNwOy4k1 8hBkg6uS9xOLez6Eqo12eYrq9RyjnUqSgQKkIO/rWBclehtMGnaF1ZVjfOTx+TcmQ X-Received: by 2002:a05:620a:7f2:b0:6ce:b70b:7d63 with SMTP id k18-20020a05620a07f200b006ceb70b7d63mr17700871qkk.770.1663714274097; Tue, 20 Sep 2022 15:51:14 -0700 (PDT) X-Google-Smtp-Source: AMsMyM4M2xUdjJeQK3N5flhmh8c8T3sswvNHuESWoByw2Y/CnkXt/nFq9WZ3AQcnWpdNgG5XBj5JKQ== X-Received: by 2002:a05:620a:7f2:b0:6ce:b70b:7d63 with SMTP id k18-20020a05620a07f200b006ceb70b7d63mr17700856qkk.770.1663714273813; Tue, 20 Sep 2022 15:51:13 -0700 (PDT) From: Peter Xu To: qemu-devel@nongnu.org Cc: "Dr . David Alan Gilbert" , Manish Mishra , peterx@redhat.com, Juan Quintela , Leonardo Bras Soares Passos , ani@anisinha.ca, "Daniel P . Berrange" Subject: [PATCH 04/14] migration: Remove RAMState.f references in compression code Date: Tue, 20 Sep 2022 18:50:56 -0400 Message-Id: <20220920225106.48451-5-peterx@redhat.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20220920225106.48451-1-peterx@redhat.com> References: <20220920225106.48451-1-peterx@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=peterx@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1663721688047100001 Content-Type: text/plain; charset="utf-8" Removing referencing to RAMState.f in compress_page_with_multi_thread() and flush_compressed_data(). Compression code by default isn't compatible with having >1 channels (or it won't currently know which channel to flush the compressed data), so to make it simple we always flush on the default to_dst_file port until someone wants to add >1 ports support, as rs->f right now can really change (after postcopy preempt is introduced). There should be no functional change at all after patch applied, since as long as rs->f referenced in compression code, it must be to_dst_file. Signed-off-by: Peter Xu Reviewed-by: Dr. David Alan Gilbert --- migration/ram.c | 12 +++++++----- 1 file changed, 7 insertions(+), 5 deletions(-) diff --git a/migration/ram.c b/migration/ram.c index 62ff2c1469..8303252b6d 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -1461,6 +1461,7 @@ static bool save_page_use_compression(RAMState *rs); =20 static void flush_compressed_data(RAMState *rs) { + MigrationState *ms =3D migrate_get_current(); int idx, len, thread_count; =20 if (!save_page_use_compression(rs)) { @@ -1479,7 +1480,7 @@ static void flush_compressed_data(RAMState *rs) for (idx =3D 0; idx < thread_count; idx++) { qemu_mutex_lock(&comp_param[idx].mutex); if (!comp_param[idx].quit) { - len =3D qemu_put_qemu_file(rs->f, comp_param[idx].file); + len =3D qemu_put_qemu_file(ms->to_dst_file, comp_param[idx].fi= le); /* * it's safe to fetch zero_page without holding comp_done_lock * as there is no further request submitted to the thread, @@ -1498,11 +1499,11 @@ static inline void set_compress_params(CompressPara= m *param, RAMBlock *block, param->offset =3D offset; } =20 -static int compress_page_with_multi_thread(RAMState *rs, RAMBlock *block, - ram_addr_t offset) +static int compress_page_with_multi_thread(RAMBlock *block, ram_addr_t off= set) { int idx, thread_count, bytes_xmit =3D -1, pages =3D -1; bool wait =3D migrate_compress_wait_thread(); + MigrationState *ms =3D migrate_get_current(); =20 thread_count =3D migrate_compress_threads(); qemu_mutex_lock(&comp_done_lock); @@ -1510,7 +1511,8 @@ retry: for (idx =3D 0; idx < thread_count; idx++) { if (comp_param[idx].done) { comp_param[idx].done =3D false; - bytes_xmit =3D qemu_put_qemu_file(rs->f, comp_param[idx].file); + bytes_xmit =3D qemu_put_qemu_file(ms->to_dst_file, + comp_param[idx].file); qemu_mutex_lock(&comp_param[idx].mutex); set_compress_params(&comp_param[idx], block, offset); qemu_cond_signal(&comp_param[idx].cond); @@ -2263,7 +2265,7 @@ static bool save_compress_page(RAMState *rs, RAMBlock= *block, ram_addr_t offset) return false; } =20 - if (compress_page_with_multi_thread(rs, block, offset) > 0) { + if (compress_page_with_multi_thread(block, offset) > 0) { return true; } =20 --=20 2.32.0 From nobody Fri Apr 26 23:13:09 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1663721895; cv=none; d=zohomail.com; s=zohoarc; b=ejYaRx6L2ykx0QhKKemXSvvTGgg1JjAB2l6VJK9AIXORjZ8DoACRGWGWKSncpfx6RW8ONM5kKQ0kL7YdrZWBX4Ru+Fa2M5LA0qjGoiO24x49LW+MQRr/nvZGQbMyns+2BdbnCcmkvLbNKCuBECRm+nj6BHGtwzakTdqKvtZPhZI= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1663721895; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=gj0iWJQkmOoJ+o3+SQ0GN4gCpxOsbiA4mOzqzJHbSgA=; b=Xvn0yHNdx7c1FKO35bAbreJD5z/rHGK9agwt9SOp35+VVJjhyaD4wc4HBIx9oN/9i1QM4IW0UjSG6jcbTjOMZdnfR1HqxIsfV32E1rEv0/vNR5Ul/dXYfYaErATIeNP9ymqyVsignKbdyZsNUrLCKHWlp1T5vIKbaxyEvySSyMY= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1663721895071441.49801019862934; Tue, 20 Sep 2022 17:58:15 -0700 (PDT) Received: from localhost ([::1]:47706 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1oao45-000708-Qr for importer@patchew.org; Tue, 20 Sep 2022 20:58:13 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:46354) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1oam6J-0005W2-CP for qemu-devel@nongnu.org; Tue, 20 Sep 2022 18:52:23 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:47353) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1oam6E-0001Tz-0C for qemu-devel@nongnu.org; Tue, 20 Sep 2022 18:52:23 -0400 Received: from mail-qt1-f200.google.com (mail-qt1-f200.google.com [209.85.160.200]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_128_GCM_SHA256) id us-mta-624-FNgCS9riNEKPZSDK3JB2hg-1; Tue, 20 Sep 2022 18:52:13 -0400 Received: by mail-qt1-f200.google.com with SMTP id n16-20020ac85b50000000b0035ce9e26eb1so2881672qtw.8 for ; Tue, 20 Sep 2022 15:52:13 -0700 (PDT) Received: from localhost.localdomain (bras-base-aurron9127w-grc-46-70-31-27-79.dsl.bell.ca. [70.31.27.79]) by smtp.gmail.com with ESMTPSA id bk14-20020a05620a1a0e00b006b5df4d2c81sm611226qkb.94.2022.09.20.15.52.11 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Tue, 20 Sep 2022 15:52:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1663714334; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=gj0iWJQkmOoJ+o3+SQ0GN4gCpxOsbiA4mOzqzJHbSgA=; b=Cvwg99MCh0r4Jqfw2FEGpKlwDSK3CDY08blDOnrWI2AjzmspVv7ySviePmA0VHyNRKbec9 Zrb6ZSxPxC/c3pqIsiQmpF8ZdBEINhRYkcCMR9GpVF0QrrvsYnocWeO4LealpD+NjdRyvz Wuwa9rVLir4xLfaqMe3YdWH4T/0G8MM= X-MC-Unique: FNgCS9riNEKPZSDK3JB2hg-1 X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date; bh=gj0iWJQkmOoJ+o3+SQ0GN4gCpxOsbiA4mOzqzJHbSgA=; b=n1MsanUM9c8aiLA9bvYix86EeOBg2WLp3TLlXiuJE3pzxaYCcW6VTenOC8RhhK8Lx9 Pc91PaqLGbN/eOSlc2f8h0x95GYqcjl9KKQCPHA55VSghulEm3KOIrHhRCE6DMvvkyS1 LeyDHu6As+gTFrPbZmVEYpPOB4RD1cA/jHvMniTPJ7HYdps8Sc3+MmQRZfyALwUbqpX4 FbO8/NJrjEAaEOfBW/EvlVGXnnIxc20kvO8MNQOr4VhlFom985aZ1OR35H3gHVhg7vWG utlI/aCGjAdheUelfuQ9fbUWqY6xvgHOuetDCpOSkBMqOwpnu7ugNTnE69YY2SCbHF7r JvCA== X-Gm-Message-State: ACrzQf2uXo3AxiSv2lxjf+VbinELjH7cY6czN5u2wNkPsm7DzK5mXQ1c bdyJbpVj9nTX95uI3sXd1yXBTjSh468zanYR7E2ppPjKYC035sMo/reKHWQJY9t/+iR8lwXB34z 15M2kP2cAvQrk92EmGIXraPP7x7L2A2pzNn4I1Z780E1I5DI4NzdN25ZJfj59UqEg X-Received: by 2002:a05:620a:2b86:b0:6ce:ee47:f733 with SMTP id dz6-20020a05620a2b8600b006ceee47f733mr11574412qkb.398.1663714332918; Tue, 20 Sep 2022 15:52:12 -0700 (PDT) X-Google-Smtp-Source: AMsMyM6Ux5Y7oOKfto9JJ+ql2Ln3pKgMnfbk5BzAHoYDtF/h5+SKSKpsrv3Ka5Ii/0QVPrgG+QxoIg== X-Received: by 2002:a05:620a:2b86:b0:6ce:ee47:f733 with SMTP id dz6-20020a05620a2b8600b006ceee47f733mr11574397qkb.398.1663714332616; Tue, 20 Sep 2022 15:52:12 -0700 (PDT) From: Peter Xu To: qemu-devel@nongnu.org Cc: peterx@redhat.com, Manish Mishra , Juan Quintela , ani@anisinha.ca, Leonardo Bras Soares Passos , "Dr . David Alan Gilbert" , "Daniel P . Berrange" Subject: [PATCH 05/14] migration: Yield bitmap_mutex properly when sending/sleeping Date: Tue, 20 Sep 2022 18:52:10 -0400 Message-Id: <20220920225210.48732-1-peterx@redhat.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20220920225106.48451-1-peterx@redhat.com> References: <20220920225106.48451-1-peterx@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=peterx@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1663721897055100001 Content-Type: text/plain; charset="utf-8" Don't take the bitmap mutex when sending pages, or when being throttled by migration_rate_limit() (which is a bit tricky to call it here in ram code, but seems still helpful). It prepares for the possibility of concurrently sending pages in >1 threads using the function ram_save_host_page() because all threads may need the bitmap_mutex to operate on bitmaps, so that either sendmsg() or any kind of qemu_sem_wait() blocking for one thread will not block the other from progressing. Signed-off-by: Peter Xu --- migration/ram.c | 42 +++++++++++++++++++++++++++++++----------- 1 file changed, 31 insertions(+), 11 deletions(-) diff --git a/migration/ram.c b/migration/ram.c index 8303252b6d..6e7de6087a 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -2463,6 +2463,7 @@ static void postcopy_preempt_reset_channel(RAMState *= rs) */ static int ram_save_host_page(RAMState *rs, PageSearchStatus *pss) { + bool page_dirty, release_lock =3D postcopy_preempt_active(); int tmppages, pages =3D 0; size_t pagesize_bits =3D qemu_ram_pagesize(pss->block) >> TARGET_PAGE_BITS; @@ -2486,22 +2487,41 @@ static int ram_save_host_page(RAMState *rs, PageSea= rchStatus *pss) break; } =20 + page_dirty =3D migration_bitmap_clear_dirty(rs, pss->block, pss->p= age); + /* + * Properly yield the lock only in postcopy preempt mode because + * both migration thread and rp-return thread can operate on the + * bitmaps. + */ + if (release_lock) { + qemu_mutex_unlock(&rs->bitmap_mutex); + } + /* Check the pages is dirty and if it is send it */ - if (migration_bitmap_clear_dirty(rs, pss->block, pss->page)) { + if (page_dirty) { tmppages =3D ram_save_target_page(rs, pss); - if (tmppages < 0) { - return tmppages; + if (tmppages >=3D 0) { + pages +=3D tmppages; + /* + * Allow rate limiting to happen in the middle of huge pag= es if + * something is sent in the current iteration. + */ + if (pagesize_bits > 1 && tmppages > 0) { + migration_rate_limit(); + } } + } else { + tmppages =3D 0; + } =20 - pages +=3D tmppages; - /* - * Allow rate limiting to happen in the middle of huge pages if - * something is sent in the current iteration. - */ - if (pagesize_bits > 1 && tmppages > 0) { - migration_rate_limit(); - } + if (release_lock) { + qemu_mutex_lock(&rs->bitmap_mutex); } + + if (tmppages < 0) { + return tmppages; + } + pss->page =3D migration_bitmap_find_dirty(rs, pss->block, pss->pag= e); } while ((pss->page < hostpage_boundary) && offset_in_ramblock(pss->block, --=20 2.32.0 From nobody Fri Apr 26 23:13:09 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1663721814; cv=none; d=zohomail.com; s=zohoarc; b=dGapO+G9gblV3hrNAK7pjjw1UHln63axco+iD5edfqGbTHi6JNkDZNd0dtuSdvLUYyaSeo/A/rgzMJyYMU28cEgCdw322FR0qufY11HS3jCclwKY8VhUGuzDlU9KunD/Hg399pz/BRzhK9InYkMypBiMj4riCChk65sqlI4BSxc= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1663721814; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=VIE9Eu3rV4oMGlwjU8nKQXaX/q5T63VbnqoTQtYRVF8=; b=dEBOHEaTDG+krsZzldUFvmIt6Ma5BliHSXjBxZuKl3/qRRtpg+YyG+kEUgSYiVp8/ZG2K3iZnlMpUnT57h9BUFBhBH31n034sjqthtskqkTm4tva9dPifgBrDBpqnvIuFln2vf8Eu/mjg9EhoO78dyXNXjvdFeLc5zs/2nVAvT8= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1663721814119140.13452908694182; Tue, 20 Sep 2022 17:56:54 -0700 (PDT) Received: from localhost ([::1]:35924 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1oao2l-00056p-KL for importer@patchew.org; Tue, 20 Sep 2022 20:56:51 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:46350) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1oam6G-0005Lj-O8 for qemu-devel@nongnu.org; Tue, 20 Sep 2022 18:52:20 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]:31162) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1oam6E-0001U3-3S for qemu-devel@nongnu.org; Tue, 20 Sep 2022 18:52:19 -0400 Received: from mail-qt1-f198.google.com (mail-qt1-f198.google.com [209.85.160.198]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_128_GCM_SHA256) id us-mta-237-XH2fB0rvPQ2WWFKS-S4Hvg-1; Tue, 20 Sep 2022 18:52:16 -0400 Received: by mail-qt1-f198.google.com with SMTP id g21-20020ac87d15000000b0035bb6f08778so2898307qtb.2 for ; Tue, 20 Sep 2022 15:52:15 -0700 (PDT) Received: from localhost.localdomain (bras-base-aurron9127w-grc-46-70-31-27-79.dsl.bell.ca. [70.31.27.79]) by smtp.gmail.com with ESMTPSA id e3-20020ac84e43000000b0031f0b43629dsm619185qtw.23.2022.09.20.15.52.13 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Tue, 20 Sep 2022 15:52:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1663714337; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=VIE9Eu3rV4oMGlwjU8nKQXaX/q5T63VbnqoTQtYRVF8=; b=RDwBmEvjkCaDKO8IrrDY39Icceom3FJlk9lR3gllWJ1kgGmSGmp6cLIqYGdSobZ2E/oJy2 PV9oc5FeYmVldBIk5AywXgo92vXExYRNqdzzyhvRPzdIqJ1fGcFp8ecsSyTtigSKXxsnJZ LSuOnTKnp9PKAZZq3Iwb2lA0nf2xvy8= X-MC-Unique: XH2fB0rvPQ2WWFKS-S4Hvg-1 X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date; bh=VIE9Eu3rV4oMGlwjU8nKQXaX/q5T63VbnqoTQtYRVF8=; b=QxDUL5I1DfeiGay8fwNDtcR/k7fz9GSurovmgM6/tbzImzUesh9GyYnqZokVmU4EoZ GMBmsf7SN/1UwCHKRzEiHld6XwUXCOLkGwBufZrcl36G/eZ1z7QpIqw2IL+PtFujDm4x d7Oonq7lqAEhqK1RapJXr8CkQwQ0OZlKHW9TTryRZIm9txDjcEEUUkoTN9FJdPpL6Csc 6RLSsIIFR6j0O1DKY6zti2nRH/GgV+2F2LvJzkQ6UJt5VezNVY4pXNVi9Bj2kSzykaol jOq18C1mzkWOKalJrM7+AFHSJZvb3ASUNwKlbqoc1mhkmQLCDeAxQN+UmaDJawawJIc1 wf8A== X-Gm-Message-State: ACrzQf0PoDCywl8UuYfWc6zx38hHOr7jTDdqxwGQ17YobYibrNu5M7SK 8fokjKO5vbJFIEys3TdvFyGqaT5p9mW7jalJju8UCJEtOcUD7VKf7WyMowxB1t1L4t2KTJYlO+9 aVbeNiSydwVZhPvpbBznTpigBI1s330Wvep10l7jSx3fMKK7VUDsQOVJQVdqbR4v3 X-Received: by 2002:a05:620a:22c5:b0:6ce:8e1:fd8c with SMTP id o5-20020a05620a22c500b006ce08e1fd8cmr18364903qki.393.1663714335307; Tue, 20 Sep 2022 15:52:15 -0700 (PDT) X-Google-Smtp-Source: AMsMyM5spK43AIwkYj0RqUP5nFBePEnJ/WNweV5FDkEcZBAFmE9yjH9h6nnQGReocljTXLgxptgQtA== X-Received: by 2002:a05:620a:22c5:b0:6ce:8e1:fd8c with SMTP id o5-20020a05620a22c500b006ce08e1fd8cmr18364884qki.393.1663714335011; Tue, 20 Sep 2022 15:52:15 -0700 (PDT) From: Peter Xu To: qemu-devel@nongnu.org Cc: peterx@redhat.com, Manish Mishra , Juan Quintela , ani@anisinha.ca, Leonardo Bras Soares Passos , "Dr . David Alan Gilbert" , "Daniel P . Berrange" Subject: [PATCH 06/14] migration: Use atomic ops properly for page accountings Date: Tue, 20 Sep 2022 18:52:12 -0400 Message-Id: <20220920225212.48785-1-peterx@redhat.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20220920225106.48451-1-peterx@redhat.com> References: <20220920225106.48451-1-peterx@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=peterx@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1663721814702100001 Content-Type: text/plain; charset="utf-8" To prepare for thread-safety on page accountings, at least below counters need to be accessed only atomically, they are: ram_counters.transferred ram_counters.duplicate ram_counters.normal ram_counters.postcopy_bytes There are a lot of other counters but they won't be accessed outside migration thread, then they're still safe to be accessed without atomic ops. Signed-off-by: Peter Xu Reviewed-by: Dr. David Alan Gilbert --- migration/migration.c | 10 +++++----- migration/multifd.c | 2 +- migration/ram.c | 29 +++++++++++++++-------------- 3 files changed, 21 insertions(+), 20 deletions(-) diff --git a/migration/migration.c b/migration/migration.c index 07c74a79a2..0eacc0c99b 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -1048,13 +1048,13 @@ static void populate_ram_info(MigrationInfo *info, = MigrationState *s) =20 info->has_ram =3D true; info->ram =3D g_malloc0(sizeof(*info->ram)); - info->ram->transferred =3D ram_counters.transferred; + info->ram->transferred =3D qatomic_read(&ram_counters.transferred); info->ram->total =3D ram_bytes_total(); - info->ram->duplicate =3D ram_counters.duplicate; + info->ram->duplicate =3D qatomic_read(&ram_counters.duplicate); /* legacy value. It is not used anymore */ info->ram->skipped =3D 0; - info->ram->normal =3D ram_counters.normal; - info->ram->normal_bytes =3D ram_counters.normal * page_size; + info->ram->normal =3D qatomic_read(&ram_counters.normal); + info->ram->normal_bytes =3D info->ram->normal * page_size; info->ram->mbps =3D s->mbps; info->ram->dirty_sync_count =3D ram_counters.dirty_sync_count; info->ram->dirty_sync_missed_zero_copy =3D @@ -1065,7 +1065,7 @@ static void populate_ram_info(MigrationInfo *info, Mi= grationState *s) info->ram->pages_per_second =3D s->pages_per_second; info->ram->precopy_bytes =3D ram_counters.precopy_bytes; info->ram->downtime_bytes =3D ram_counters.downtime_bytes; - info->ram->postcopy_bytes =3D ram_counters.postcopy_bytes; + info->ram->postcopy_bytes =3D qatomic_read(&ram_counters.postcopy_byte= s); =20 if (migrate_use_xbzrle()) { info->has_xbzrle_cache =3D true; diff --git a/migration/multifd.c b/migration/multifd.c index 586ddc9d65..460326acd4 100644 --- a/migration/multifd.c +++ b/migration/multifd.c @@ -437,7 +437,7 @@ static int multifd_send_pages(QEMUFile *f) + p->packet_len; qemu_file_acct_rate_limit(f, transferred); ram_counters.multifd_bytes +=3D transferred; - ram_counters.transferred +=3D transferred; + qatomic_add(&ram_counters.transferred, transferred); qemu_mutex_unlock(&p->mutex); qemu_sem_post(&p->sem); =20 diff --git a/migration/ram.c b/migration/ram.c index 6e7de6087a..5bd3d76bf0 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -432,11 +432,11 @@ static void ram_transferred_add(uint64_t bytes) if (runstate_is_running()) { ram_counters.precopy_bytes +=3D bytes; } else if (migration_in_postcopy()) { - ram_counters.postcopy_bytes +=3D bytes; + qatomic_add(&ram_counters.postcopy_bytes, bytes); } else { ram_counters.downtime_bytes +=3D bytes; } - ram_counters.transferred +=3D bytes; + qatomic_add(&ram_counters.transferred, bytes); } =20 void dirty_sync_missed_zero_copy(void) @@ -725,7 +725,7 @@ void mig_throttle_counter_reset(void) =20 rs->time_last_bitmap_sync =3D qemu_clock_get_ms(QEMU_CLOCK_REALTIME); rs->num_dirty_pages_period =3D 0; - rs->bytes_xfer_prev =3D ram_counters.transferred; + rs->bytes_xfer_prev =3D qatomic_read(&ram_counters.transferred); } =20 /** @@ -1085,8 +1085,9 @@ uint64_t ram_pagesize_summary(void) =20 uint64_t ram_get_total_transferred_pages(void) { - return ram_counters.normal + ram_counters.duplicate + - compression_counters.pages + xbzrle_counters.pages; + return qatomic_read(&ram_counters.normal) + + qatomic_read(&ram_counters.duplicate) + + compression_counters.pages + xbzrle_counters.pages; } =20 static void migration_update_rates(RAMState *rs, int64_t end_time) @@ -1145,8 +1146,8 @@ static void migration_trigger_throttle(RAMState *rs) { MigrationState *s =3D migrate_get_current(); uint64_t threshold =3D s->parameters.throttle_trigger_threshold; - - uint64_t bytes_xfer_period =3D ram_counters.transferred - rs->bytes_xf= er_prev; + uint64_t bytes_xfer_period =3D + qatomic_read(&ram_counters.transferred) - rs->bytes_xfer_prev; uint64_t bytes_dirty_period =3D rs->num_dirty_pages_period * TARGET_PA= GE_SIZE; uint64_t bytes_dirty_threshold =3D bytes_xfer_period * threshold / 100; =20 @@ -1285,7 +1286,7 @@ static int save_zero_page(RAMState *rs, RAMBlock *blo= ck, ram_addr_t offset) int len =3D save_zero_page_to_file(rs, rs->f, block, offset); =20 if (len) { - ram_counters.duplicate++; + qatomic_inc(&ram_counters.duplicate); ram_transferred_add(len); return 1; } @@ -1322,9 +1323,9 @@ static bool control_save_page(RAMState *rs, RAMBlock = *block, ram_addr_t offset, } =20 if (bytes_xmit > 0) { - ram_counters.normal++; + qatomic_inc(&ram_counters.normal); } else if (bytes_xmit =3D=3D 0) { - ram_counters.duplicate++; + qatomic_inc(&ram_counters.duplicate); } =20 return true; @@ -1354,7 +1355,7 @@ static int save_normal_page(RAMState *rs, RAMBlock *b= lock, ram_addr_t offset, qemu_put_buffer(rs->f, buf, TARGET_PAGE_SIZE); } ram_transferred_add(TARGET_PAGE_SIZE); - ram_counters.normal++; + qatomic_inc(&ram_counters.normal); return 1; } =20 @@ -1448,7 +1449,7 @@ update_compress_thread_counts(const CompressParam *pa= ram, int bytes_xmit) ram_transferred_add(bytes_xmit); =20 if (param->zero_page) { - ram_counters.duplicate++; + qatomic_inc(&ram_counters.duplicate); return; } =20 @@ -2620,9 +2621,9 @@ void acct_update_position(QEMUFile *f, size_t size, b= ool zero) uint64_t pages =3D size / TARGET_PAGE_SIZE; =20 if (zero) { - ram_counters.duplicate +=3D pages; + qatomic_add(&ram_counters.duplicate, pages); } else { - ram_counters.normal +=3D pages; + qatomic_add(&ram_counters.normal, pages); ram_transferred_add(size); qemu_file_credit_transfer(f, size); } --=20 2.32.0 From nobody Fri Apr 26 23:13:09 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1663719267; cv=none; d=zohomail.com; s=zohoarc; b=Sb1cYj+zX6umAUdNXb+WpX9g8l226K/ZIz/aAfNXa02ud53Y10WMb7bVgDltnUzues/5BnA/d7YWYPN3tHGvu4ikaZ9585luOV75JVIUV+MHyxdC9Qr8vfto5DjKNQIotWctduflh4KBCLmhrx6CHyL8EEtreRi5wWNBrN306FM= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1663719267; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=t3jhrWF4/v7ccWm56MSuSsY4ob3N3Q5AcO537SH6ikw=; b=M6XBQyqqU1bJ/eNqQdTa6KW5UMdzB2fbTZV5S5c5O+XUjYH48p7uvqO6Uod4HiJjV0zfKWOxy6UIkPA/ySy4yAbk2mj8np8T9QWxO6zoNYXItfluQZPiHiTK9d7OG48YVe9005t2LZ7RAZi/1DAYYcbRpqmGKKsEMY71GVRKoC0= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1663719267924327.4983457712086; Tue, 20 Sep 2022 17:14:27 -0700 (PDT) Received: from localhost ([::1]:53280 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1oanNi-0000AD-TS for importer@patchew.org; Tue, 20 Sep 2022 20:14:26 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:46352) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1oam6I-0005UO-Vt for qemu-devel@nongnu.org; Tue, 20 Sep 2022 18:52:23 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]:42629) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1oam6G-0001UE-Uj for qemu-devel@nongnu.org; Tue, 20 Sep 2022 18:52:22 -0400 Received: from mail-qt1-f197.google.com (mail-qt1-f197.google.com [209.85.160.197]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_128_GCM_SHA256) id us-mta-331-Of0jVtR4Mhe218VLmlM5Kw-1; Tue, 20 Sep 2022 18:52:18 -0400 Received: by mail-qt1-f197.google.com with SMTP id w4-20020a05622a134400b0035cbc5ec9a2so2911563qtk.14 for ; Tue, 20 Sep 2022 15:52:18 -0700 (PDT) Received: from localhost.localdomain (bras-base-aurron9127w-grc-46-70-31-27-79.dsl.bell.ca. [70.31.27.79]) by smtp.gmail.com with ESMTPSA id dt9-20020a05620a478900b006cddf59a600sm704164qkb.34.2022.09.20.15.52.16 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Tue, 20 Sep 2022 15:52:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1663714340; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=t3jhrWF4/v7ccWm56MSuSsY4ob3N3Q5AcO537SH6ikw=; b=CFq2dJBoCO8XVLL4CXNsEIUGSM3fcgGY/eg/WD3Ldhy9Sj9sJ7qFj8C3SGQjvNmYXL+99/ N7eRd4HHu0R+7MTtt21FY2g2ORlwmCU88u5YiN7mmfoZLsKr13g2xNrXTc7tmAl8CqRjRV E/fkUPoPN+ycaE2YzHtgjQD8CRoNL3w= X-MC-Unique: Of0jVtR4Mhe218VLmlM5Kw-1 X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date; bh=t3jhrWF4/v7ccWm56MSuSsY4ob3N3Q5AcO537SH6ikw=; b=3T07NYPyoWSDDPiyGudbDcZmCqTz+a7kQKlLRroPZa0hP94FdBlzB07iX05BWAnD6J 0VS7hDWeKoYuDCuTli6r5QhqxMcnnoSQUHPGdvzbp1wnhK86zEpNIjJ2+gV4M0ovzy+P 74jOiiY9k6PYYKeWDSNXRpXsHSpxDrY7NH1BbXLldGbfptusojLxXEiEOtaexdtWPpsA j/DL5nRg0gvf9/Qm/eGBIUqJ/ie3dqDGMKYKfb4z0mwDJhK7Z1AEbr8WHJ9blMKzdBTr ZqV4O/ymr2xAwkp/d0RVJLeM1lti5jpLXuXzd+g7QhvictCQjn8DQ8C18i6/g/9+KhN5 dKDQ== X-Gm-Message-State: ACrzQf3TD+8mWMrWnrU9+Xr5FZ3uQjLZrIM32SydnuEwSvN/44cNNrva 8mzwORgsKzwAfC1LaeMwn9qrJMJvMF51oxaO2IjvWIRn/Alq3fQgw3iuknFJ4Ry9IIQJbKR1BWn gKoYxH3/Cdn6YZGjPXTnQTw+6m1/CFKnpQ7WwfOcsxnEcnzbnaVzKFvYYnuNDlvzd X-Received: by 2002:a05:620a:294a:b0:6ce:9944:f65a with SMTP id n10-20020a05620a294a00b006ce9944f65amr18453311qkp.434.1663714337888; Tue, 20 Sep 2022 15:52:17 -0700 (PDT) X-Google-Smtp-Source: AMsMyM4tieH0RYGmotsYVkTehGGCn3lwCyfTXlvpZUO8+JnIAJ3x9dXwtY0izsTvY7CXXeHfF3MbBA== X-Received: by 2002:a05:620a:294a:b0:6ce:9944:f65a with SMTP id n10-20020a05620a294a00b006ce9944f65amr18453277qkp.434.1663714337046; Tue, 20 Sep 2022 15:52:17 -0700 (PDT) From: Peter Xu To: qemu-devel@nongnu.org Cc: peterx@redhat.com, Manish Mishra , Juan Quintela , ani@anisinha.ca, Leonardo Bras Soares Passos , "Dr . David Alan Gilbert" , "Daniel P . Berrange" Subject: [PATCH 07/14] migration: Teach PSS about host page Date: Tue, 20 Sep 2022 18:52:15 -0400 Message-Id: <20220920225215.48840-1-peterx@redhat.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20220920225106.48451-1-peterx@redhat.com> References: <20220920225106.48451-1-peterx@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=peterx@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1663719268504100001 Content-Type: text/plain; charset="utf-8" Migration code has a lot to do with host pages. Teaching PSS core about the idea of host page helps a lot and makes the code clean. Meanwhile, this prepares for the future changes that can leverage the new PSS helpers that this patch introduces to send host page in another thread. Three more fields are introduced for this: (1) host_page_sending: this is set to true when QEMU is sending a host page, false otherwise. (2) host_page_{start|end}: these point to the start/end of host page we're sending, and it's only valid when host_page_sending=3D=3Dtrue. For example, when we look up the next dirty page on the ramblock, with host_page_sending=3D=3Dtrue, we'll not try to look for anything beyond the current host page boundary. This can be slightly efficient than current code because currently we'll set pss->page to next dirty bit (which can be over current host page boundary) and reset it to host page boundary if we found it goes beyond that. With above, we can easily make migration_bitmap_find_dirty() self contained by updating pss->page properly. rs* parameter is removed because it's not even used in old code. When sending a host page, we should use the pss helpers like this: - pss_host_page_prepare(pss): called before sending host page - pss_within_range(pss): whether we're still working on the cur host page? - pss_host_page_finish(pss): called after sending a host page Then we can use ram_save_target_page() to save one small page. Currently ram_save_host_page() is still the only user. If there'll be another function to send host page (e.g. in return path thread) in the future, it should follow the same style. Signed-off-by: Peter Xu Reviewed-by: Dr. David Alan Gilbert --- migration/ram.c | 95 +++++++++++++++++++++++++++++++++++++++---------- 1 file changed, 76 insertions(+), 19 deletions(-) diff --git a/migration/ram.c b/migration/ram.c index 5bd3d76bf0..3f720b6de2 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -474,6 +474,11 @@ struct PageSearchStatus { * postcopy pages via postcopy preempt channel. */ bool postcopy_target_channel; + /* Whether we're sending a host page */ + bool host_page_sending; + /* The start/end of current host page. Only valid if host_page_sendin= g=3D=3Dtrue */ + unsigned long host_page_start; + unsigned long host_page_end; }; typedef struct PageSearchStatus PageSearchStatus; =20 @@ -851,26 +856,38 @@ static int save_xbzrle_page(RAMState *rs, uint8_t **c= urrent_data, } =20 /** - * migration_bitmap_find_dirty: find the next dirty page from start + * pss_find_next_dirty: find the next dirty page of current ramblock * - * Returns the page offset within memory region of the start of a dirty pa= ge + * This function updates pss->page to point to the next dirty page index + * within the ramblock to migrate, or the end of ramblock when nothing + * found. Note that when pss->host_page_sending=3D=3Dtrue it means we're + * during sending a host page, so we won't look for dirty page that is + * outside the host page boundary. * - * @rs: current RAM state - * @rb: RAMBlock where to search for dirty pages - * @start: page where we start the search + * @pss: the current page search status */ -static inline -unsigned long migration_bitmap_find_dirty(RAMState *rs, RAMBlock *rb, - unsigned long start) +static void pss_find_next_dirty(PageSearchStatus *pss) { + RAMBlock *rb =3D pss->block; unsigned long size =3D rb->used_length >> TARGET_PAGE_BITS; unsigned long *bitmap =3D rb->bmap; =20 if (ramblock_is_ignored(rb)) { - return size; + /* Points directly to the end, so we know no dirty page */ + pss->page =3D size; + return; + } + + /* + * If during sending a host page, only look for dirty pages within the + * current host page being send. + */ + if (pss->host_page_sending) { + assert(pss->host_page_end); + size =3D MIN(size, pss->host_page_end); } =20 - return find_next_bit(bitmap, size, start); + pss->page =3D find_next_bit(bitmap, size, pss->page); } =20 static void migration_clear_memory_region_dirty_bitmap(RAMBlock *rb, @@ -1556,7 +1573,9 @@ static bool find_dirty_block(RAMState *rs, PageSearch= Status *pss, bool *again) pss->postcopy_requested =3D false; pss->postcopy_target_channel =3D RAM_CHANNEL_PRECOPY; =20 - pss->page =3D migration_bitmap_find_dirty(rs, pss->block, pss->page); + /* Update pss->page for the next dirty bit in ramblock */ + pss_find_next_dirty(pss); + if (pss->complete_round && pss->block =3D=3D rs->last_seen_block && pss->page >=3D rs->last_page) { /* @@ -2446,6 +2465,44 @@ static void postcopy_preempt_reset_channel(RAMState = *rs) } } =20 +/* Should be called before sending a host page */ +static void pss_host_page_prepare(PageSearchStatus *pss) +{ + /* How many guest pages are there in one host page? */ + size_t guest_pfns =3D qemu_ram_pagesize(pss->block) >> TARGET_PAGE_BIT= S; + + pss->host_page_sending =3D true; + pss->host_page_start =3D ROUND_DOWN(pss->page, guest_pfns); + pss->host_page_end =3D ROUND_UP(pss->page + 1, guest_pfns); +} + +/* + * Whether the page pointed by PSS is within the host page being sent. + * Must be called after a previous pss_host_page_prepare(). + */ +static bool pss_within_range(PageSearchStatus *pss) +{ + ram_addr_t ram_addr; + + assert(pss->host_page_sending); + + /* Over host-page boundary? */ + if (pss->page >=3D pss->host_page_end) { + return false; + } + + ram_addr =3D ((ram_addr_t)pss->page) << TARGET_PAGE_BITS; + + return offset_in_ramblock(pss->block, ram_addr); +} + +static void pss_host_page_finish(PageSearchStatus *pss) +{ + pss->host_page_sending =3D false; + /* This is not needed, but just to reset it */ + pss->host_page_start =3D pss->host_page_end =3D 0; +} + /** * ram_save_host_page: save a whole host page * @@ -2468,8 +2525,6 @@ static int ram_save_host_page(RAMState *rs, PageSearc= hStatus *pss) int tmppages, pages =3D 0; size_t pagesize_bits =3D qemu_ram_pagesize(pss->block) >> TARGET_PAGE_BITS; - unsigned long hostpage_boundary =3D - QEMU_ALIGN_UP(pss->page + 1, pagesize_bits); unsigned long start_page =3D pss->page; int res; =20 @@ -2482,6 +2537,9 @@ static int ram_save_host_page(RAMState *rs, PageSearc= hStatus *pss) postcopy_preempt_choose_channel(rs, pss); } =20 + /* Update host page boundary information */ + pss_host_page_prepare(pss); + do { if (postcopy_needs_preempt(rs, pss)) { postcopy_do_preempt(rs, pss); @@ -2520,15 +2578,14 @@ static int ram_save_host_page(RAMState *rs, PageSea= rchStatus *pss) } =20 if (tmppages < 0) { + pss_host_page_finish(pss); return tmppages; } =20 - pss->page =3D migration_bitmap_find_dirty(rs, pss->block, pss->pag= e); - } while ((pss->page < hostpage_boundary) && - offset_in_ramblock(pss->block, - ((ram_addr_t)pss->page) << TARGET_PAGE_BIT= S)); - /* The offset we leave with is the min boundary of host page and block= */ - pss->page =3D MIN(pss->page, hostpage_boundary); + pss_find_next_dirty(pss); + } while (pss_within_range(pss)); + + pss_host_page_finish(pss); =20 /* * When with postcopy preempt mode, flush the data as soon as possible= for --=20 2.32.0 From nobody Fri Apr 26 23:13:09 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1663718203; cv=none; d=zohomail.com; s=zohoarc; b=kqc91f4b3Z9LQHddVh0UGAQfGYy6YNUDmNY4SNixfU9ET32BSNuGcunwwiZ83R5Zo4lJdhf4naZK31E8A0tRaYXqW2vIljgW/wC8AJhhaNmn1QCP2HSbuF7g00ZiH4cSlZE45r8o+knzTRiRCi9Y3SKO987gGao186QcXVsjG1s= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1663718203; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=xFmLqmPclxQjXvp+fo/kfStgfOruVwVxdRfu5m7Nk1Y=; b=jNSKm15DGz9RSTZ1AljXJl8LQDxCGokeexv6TzWLSD21v8+5ky2P77TlldVdtrwbmgcdg6Do4HVTe3VV6pu+rsi9Xk46GZR/Bwvjaj+5ESL4YqcugIlXT4s+oKk2yY77ad5wmfWDnn+FvdCKXIvnGW4XDMlQfVLsm71v1yPNkdI= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1663718203231144.6105070782703; Tue, 20 Sep 2022 16:56:43 -0700 (PDT) Received: from localhost ([::1]:42760 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1oan6X-0002ge-6b for importer@patchew.org; Tue, 20 Sep 2022 19:56:42 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:33448) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1oam6L-0005d4-P1 for qemu-devel@nongnu.org; Tue, 20 Sep 2022 18:52:29 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:21387) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1oam6H-0001UJ-Op for qemu-devel@nongnu.org; Tue, 20 Sep 2022 18:52:25 -0400 Received: from mail-qk1-f198.google.com (mail-qk1-f198.google.com [209.85.222.198]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_128_GCM_SHA256) id us-mta-352-sIFlipDHPZOMO1-gpVj3uQ-1; Tue, 20 Sep 2022 18:52:20 -0400 Received: by mail-qk1-f198.google.com with SMTP id i7-20020a05620a404700b006ced0d02b68so3113023qko.10 for ; Tue, 20 Sep 2022 15:52:20 -0700 (PDT) Received: from localhost.localdomain (bras-base-aurron9127w-grc-46-70-31-27-79.dsl.bell.ca. [70.31.27.79]) by smtp.gmail.com with ESMTPSA id b4-20020ac86bc4000000b0035c1e18762csm492417qtt.84.2022.09.20.15.52.18 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Tue, 20 Sep 2022 15:52:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1663714341; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=xFmLqmPclxQjXvp+fo/kfStgfOruVwVxdRfu5m7Nk1Y=; b=V0ik9ilCE7ME5PGxb32QbpbdEoblk8xFhiDPepzykFKkYstyp6+Xli4xQyMZ8bSI+Pc5gp AvhNM+XBl1235e1p/LfOtx6NSpgKJAmKwb1czMvgR4g9Klrd7R8VL6QZFWEU5k6mKeghLY FJjIJICXHQ+45zbbR5DAWE8YmVv8STg= X-MC-Unique: sIFlipDHPZOMO1-gpVj3uQ-1 X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date; bh=xFmLqmPclxQjXvp+fo/kfStgfOruVwVxdRfu5m7Nk1Y=; b=NkuSSF+oAGF+ibCDS/O6ks63CbLsPrqXwnOn6CX5xrtcaz9c6nLcsvWDY1rQtSw3mz 6c/67KrDDMbrB5ia0fn2ac7Tu/C8CyO06Sly2r4da7Tx9ShKoY8pM6XHKcGcUPe2bSUO Sulop+6U56Um0OcMkyJfr2tfWZH21h2tmkBPb4ngUfqklzckAhY5KUO8zez3KD6jfaB0 LfX5eBj+g7FO2xeukrggVBim+JhS8GooKmrDDmyxJFkWDtlHzHVLXiNNXVfD2gn/Dk5b yOLUNR8XdcG/NDEp/1HZ5Tp/pEgUTC7Y9qPM8PRoAyAb9wgrmZoyFHHJSsVBZFM//uw8 TjHw== X-Gm-Message-State: ACrzQf15xz9uSW29fOUwVc+M+49fKI5dSplMuHMJzHgKDHLh5P1Oe2e8 Dp9pSV/TOIzEZFmcg6HGnn3/yvehaU3C5sxj2XsofttYkfWM7OLQzl98AKSC8ZZcqNvirmoyODP KMBDOyk/HxeLpOndmMhPVM0o3x8X9ck6d0jilHSdFQRQFLB1kVltsh9vOzekxOASJ X-Received: by 2002:a05:620a:1a17:b0:6ce:7c1b:c27f with SMTP id bk23-20020a05620a1a1700b006ce7c1bc27fmr18546067qkb.42.1663714339377; Tue, 20 Sep 2022 15:52:19 -0700 (PDT) X-Google-Smtp-Source: AMsMyM5ugOZi7nh5jyBtQBV9vC6QzvdV4/XHu0m3IsVcVr8naTnn/lF3R8Zpfqzxn9GVeanuCFJfvw== X-Received: by 2002:a05:620a:1a17:b0:6ce:7c1b:c27f with SMTP id bk23-20020a05620a1a1700b006ce7c1bc27fmr18546037qkb.42.1663714339041; Tue, 20 Sep 2022 15:52:19 -0700 (PDT) From: Peter Xu To: qemu-devel@nongnu.org Cc: peterx@redhat.com, Manish Mishra , Juan Quintela , ani@anisinha.ca, Leonardo Bras Soares Passos , "Dr . David Alan Gilbert" , "Daniel P . Berrange" Subject: [PATCH 08/14] migration: Introduce pss_channel Date: Tue, 20 Sep 2022 18:52:17 -0400 Message-Id: <20220920225217.48893-1-peterx@redhat.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20220920225106.48451-1-peterx@redhat.com> References: <20220920225106.48451-1-peterx@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=peterx@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1663718203527100001 Content-Type: text/plain; charset="utf-8" Introduce pss_channel for PageSearchStatus, define it as "the migration channel to be used to transfer this host page". We used to have rs->f, which is a mirror to MigrationState.to_dst_file. After postcopy preempt initial version, rs->f can be dynamically changed depending on which channel we want to use. But that later work still doesn't grant full concurrency of sending pages in e.g. different threads, because rs->f can either be the PRECOPY channel or POSTCOPY channel. This needs to be per-thread too. PageSearchStatus is actually a good piece of struct which we can leverage if we want to have multiple threads sending pages. Sending a single guest page may not make sense, so we make the granule to be "host page", and in the PSS structure we allow specify a QEMUFile* to migrate a specific host page. Then we open the possibility to specify different channels in different threads with different PSS structures. The PSS prefix can be slightly misleading here because e.g. for the upcoming usage of postcopy channel/thread it's not "searching" (or, scanning) at all but sending the explicit page that was requested. However since PSS existed for some years keep it as-is until someone complains. This patch mostly (simply) replace rs->f with pss->pss_channel only. No functional change intended for this patch yet. But it does prepare to finally drop rs->f, and make ram_save_guest_page() thread safe. Signed-off-by: Peter Xu Reviewed-by: Dr. David Alan Gilbert --- migration/ram.c | 70 +++++++++++++++++++++++++++---------------------- 1 file changed, 38 insertions(+), 32 deletions(-) diff --git a/migration/ram.c b/migration/ram.c index 3f720b6de2..40ff5dc49f 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -446,6 +446,8 @@ void dirty_sync_missed_zero_copy(void) =20 /* used by the search for pages to send */ struct PageSearchStatus { + /* The migration channel used for a specific host page */ + QEMUFile *pss_channel; /* Current block being searched */ RAMBlock *block; /* Current page to search from */ @@ -768,9 +770,9 @@ static void xbzrle_cache_zero_page(RAMState *rs, ram_ad= dr_t current_addr) * @block: block that contains the page we want to send * @offset: offset inside the block for the page */ -static int save_xbzrle_page(RAMState *rs, uint8_t **current_data, - ram_addr_t current_addr, RAMBlock *block, - ram_addr_t offset) +static int save_xbzrle_page(RAMState *rs, QEMUFile *file, + uint8_t **current_data, ram_addr_t current_add= r, + RAMBlock *block, ram_addr_t offset) { int encoded_len =3D 0, bytes_xbzrle; uint8_t *prev_cached_page; @@ -838,11 +840,11 @@ static int save_xbzrle_page(RAMState *rs, uint8_t **c= urrent_data, } =20 /* Send XBZRLE based compressed page */ - bytes_xbzrle =3D save_page_header(rs, rs->f, block, + bytes_xbzrle =3D save_page_header(rs, file, block, offset | RAM_SAVE_FLAG_XBZRLE); - qemu_put_byte(rs->f, ENCODING_FLAG_XBZRLE); - qemu_put_be16(rs->f, encoded_len); - qemu_put_buffer(rs->f, XBZRLE.encoded_buf, encoded_len); + qemu_put_byte(file, ENCODING_FLAG_XBZRLE); + qemu_put_be16(file, encoded_len); + qemu_put_buffer(file, XBZRLE.encoded_buf, encoded_len); bytes_xbzrle +=3D encoded_len + 1 + 2; /* * Like compressed_size (please see update_compress_thread_counts), @@ -1298,9 +1300,10 @@ static int save_zero_page_to_file(RAMState *rs, QEMU= File *file, * @block: block that contains the page we want to send * @offset: offset inside the block for the page */ -static int save_zero_page(RAMState *rs, RAMBlock *block, ram_addr_t offset) +static int save_zero_page(RAMState *rs, QEMUFile *file, RAMBlock *block, + ram_addr_t offset) { - int len =3D save_zero_page_to_file(rs, rs->f, block, offset); + int len =3D save_zero_page_to_file(rs, file, block, offset); =20 if (len) { qatomic_inc(&ram_counters.duplicate); @@ -1317,15 +1320,15 @@ static int save_zero_page(RAMState *rs, RAMBlock *b= lock, ram_addr_t offset) * * Return true if the pages has been saved, otherwise false is returned. */ -static bool control_save_page(RAMState *rs, RAMBlock *block, ram_addr_t of= fset, - int *pages) +static bool control_save_page(PageSearchStatus *pss, RAMBlock *block, + ram_addr_t offset, int *pages) { uint64_t bytes_xmit =3D 0; int ret; =20 *pages =3D -1; - ret =3D ram_control_save_page(rs->f, block->offset, offset, TARGET_PAG= E_SIZE, - &bytes_xmit); + ret =3D ram_control_save_page(pss->pss_channel, block->offset, offset, + TARGET_PAGE_SIZE, &bytes_xmit); if (ret =3D=3D RAM_SAVE_CONTROL_NOT_SUPP) { return false; } @@ -1359,17 +1362,17 @@ static bool control_save_page(RAMState *rs, RAMBloc= k *block, ram_addr_t offset, * @buf: the page to be sent * @async: send to page asyncly */ -static int save_normal_page(RAMState *rs, RAMBlock *block, ram_addr_t offs= et, - uint8_t *buf, bool async) +static int save_normal_page(RAMState *rs, QEMUFile *file, RAMBlock *block, + ram_addr_t offset, uint8_t *buf, bool async) { - ram_transferred_add(save_page_header(rs, rs->f, block, + ram_transferred_add(save_page_header(rs, file, block, offset | RAM_SAVE_FLAG_PAGE)); if (async) { - qemu_put_buffer_async(rs->f, buf, TARGET_PAGE_SIZE, + qemu_put_buffer_async(file, buf, TARGET_PAGE_SIZE, migrate_release_ram() && migration_in_postcopy()); } else { - qemu_put_buffer(rs->f, buf, TARGET_PAGE_SIZE); + qemu_put_buffer(file, buf, TARGET_PAGE_SIZE); } ram_transferred_add(TARGET_PAGE_SIZE); qatomic_inc(&ram_counters.normal); @@ -1402,8 +1405,8 @@ static int ram_save_page(RAMState *rs, PageSearchStat= us *pss) =20 XBZRLE_cache_lock(); if (rs->xbzrle_enabled && !migration_in_postcopy()) { - pages =3D save_xbzrle_page(rs, &p, current_addr, block, - offset); + pages =3D save_xbzrle_page(rs, pss->pss_channel, &p, current_addr, + block, offset); if (!rs->last_stage) { /* Can't send this cached data async, since the cache page * might get updated before it gets to the wire @@ -1414,7 +1417,8 @@ static int ram_save_page(RAMState *rs, PageSearchStat= us *pss) =20 /* XBZRLE overflow or normal page */ if (pages =3D=3D -1) { - pages =3D save_normal_page(rs, block, offset, p, send_async); + pages =3D save_normal_page(rs, pss->pss_channel, block, offset, + p, send_async); } =20 XBZRLE_cache_unlock(); @@ -1422,10 +1426,10 @@ static int ram_save_page(RAMState *rs, PageSearchSt= atus *pss) return pages; } =20 -static int ram_save_multifd_page(RAMState *rs, RAMBlock *block, +static int ram_save_multifd_page(QEMUFile *file, RAMBlock *block, ram_addr_t offset) { - if (multifd_queue_page(rs->f, block, offset) < 0) { + if (multifd_queue_page(file, block, offset) < 0) { return -1; } ram_counters.normal++; @@ -1720,7 +1724,7 @@ static int ram_save_release_protection(RAMState *rs, = PageSearchStatus *pss, uint64_t run_length =3D (pss->page - start_page) << TARGET_PAGE_BI= TS; =20 /* Flush async buffers before un-protect. */ - qemu_fflush(rs->f); + qemu_fflush(pss->pss_channel); /* Un-protect memory range. */ res =3D uffd_change_protection(rs->uffdio_fd, page_address, run_le= ngth, false, false); @@ -2307,7 +2311,7 @@ static int ram_save_target_page(RAMState *rs, PageSea= rchStatus *pss) ram_addr_t offset =3D ((ram_addr_t)pss->page) << TARGET_PAGE_BITS; int res; =20 - if (control_save_page(rs, block, offset, &res)) { + if (control_save_page(pss, block, offset, &res)) { return res; } =20 @@ -2315,7 +2319,7 @@ static int ram_save_target_page(RAMState *rs, PageSea= rchStatus *pss) return 1; } =20 - res =3D save_zero_page(rs, block, offset); + res =3D save_zero_page(rs, pss->pss_channel, block, offset); if (res > 0) { /* Must let xbzrle know, otherwise a previous (now 0'd) cached * page would be stale @@ -2336,7 +2340,7 @@ static int ram_save_target_page(RAMState *rs, PageSea= rchStatus *pss) */ if (!save_page_use_compression(rs) && migrate_use_multifd() && !migration_in_postcopy()) { - return ram_save_multifd_page(rs, block, offset); + return ram_save_multifd_page(pss->pss_channel, block, offset); } =20 return ram_save_page(rs, pss); @@ -2533,10 +2537,6 @@ static int ram_save_host_page(RAMState *rs, PageSear= chStatus *pss) return 0; } =20 - if (postcopy_preempt_active()) { - postcopy_preempt_choose_channel(rs, pss); - } - /* Update host page boundary information */ pss_host_page_prepare(pss); =20 @@ -2597,7 +2597,7 @@ static int ram_save_host_page(RAMState *rs, PageSearc= hStatus *pss) * explicit flush or it won't flush until the buffer is full. */ if (migrate_postcopy_preempt() && pss->postcopy_requested) { - qemu_fflush(rs->f); + qemu_fflush(pss->pss_channel); } =20 res =3D ram_save_release_protection(rs, pss, start_page); @@ -2663,6 +2663,12 @@ static int ram_find_and_save_block(RAMState *rs) } =20 if (found) { + /* Update rs->f with correct channel */ + if (postcopy_preempt_active()) { + postcopy_preempt_choose_channel(rs, &pss); + } + /* Cache rs->f in pss_channel (TODO: remove rs->f) */ + pss.pss_channel =3D rs->f; pages =3D ram_save_host_page(rs, &pss); } } while (!pages && again); --=20 2.32.0 From nobody Fri Apr 26 23:13:09 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1663718769; cv=none; d=zohomail.com; s=zohoarc; b=aFGDiMVsSOvsUM432DUMB+NZjRxzumccR91iBzphrm57YwmEMaQ53H4TZIaiPaPZIwBWmRF3zd1KSmHFo3KYjcN3DZXWH5gJ9bFDgLZoaBft5jhI5Rxr/Y54p0TYIwioboEjc01sJgxH+V3d2IEc23lRBN3rep0FeqWB7ATs4Z0= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1663718769; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=QIGbU6jZWHzL+qBCAybLobJs+t2MV+5pyZn8uPH38lc=; b=Hi8WDRm8uMqV0WsDFI40XXQqzDagCjwURC3Zsr7RHdfk+JUoauPVmbUpCoIpdrAH+1J90BSMITACjrvN3KOEkT83Ym1f6IwjQanoF49k8eompqQQ+R8dNzfugLv4WLGEinRGt6ngVml+GG81nyuRZ6jt+mK+8gq+9d82sqnEUZM= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1663718769357406.98479451082403; Tue, 20 Sep 2022 17:06:09 -0700 (PDT) Received: from localhost ([::1]:40302 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1oanFe-0000zb-VE for importer@patchew.org; Tue, 20 Sep 2022 20:06:07 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:41480) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1oam6Y-0006G9-VB for qemu-devel@nongnu.org; Tue, 20 Sep 2022 18:52:38 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:44247) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1oam6J-0001UW-KP for qemu-devel@nongnu.org; Tue, 20 Sep 2022 18:52:38 -0400 Received: from mail-qt1-f200.google.com (mail-qt1-f200.google.com [209.85.160.200]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_128_GCM_SHA256) id us-mta-482-Wunzc4vKPhGF9jIGtF1hoA-1; Tue, 20 Sep 2022 18:52:21 -0400 Received: by mail-qt1-f200.google.com with SMTP id fe14-20020a05622a4d4e00b0035cc376b1d5so2895906qtb.4 for ; Tue, 20 Sep 2022 15:52:21 -0700 (PDT) Received: from localhost.localdomain (bras-base-aurron9127w-grc-46-70-31-27-79.dsl.bell.ca. [70.31.27.79]) by smtp.gmail.com with ESMTPSA id cc24-20020a05622a411800b0035ba3cae6basm569618qtb.34.2022.09.20.15.52.20 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Tue, 20 Sep 2022 15:52:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1663714342; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=QIGbU6jZWHzL+qBCAybLobJs+t2MV+5pyZn8uPH38lc=; b=e9jOaAfsEQAJ5WTB4PVEjcvFROdXweV3wVpI0Seg8o91GNvIc4fMY1xyXyPNKlsNFAsKK5 9cieETUMCvi+MCOWX/bT2Xe3WSbn/jgbWu5c68aI3dh87A+RQEnJ9x1RH/GnT+y5UwZ3PE hiTS2QILkqWKS/7Z+6IdgwJ/bHZEClU= X-MC-Unique: Wunzc4vKPhGF9jIGtF1hoA-1 X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date; bh=QIGbU6jZWHzL+qBCAybLobJs+t2MV+5pyZn8uPH38lc=; b=oDcQKvAs1GM9Es48e8LGNN/WdhDHKaROtNdrTKrUk5GvaMdz4gSn/yTb9E9MjmS+OO fS4cXfOtxFasu3O5nNwVC7ppVlHoMclINlv4Ns2KZakhJ1lMOikYOKd7ppgJSMCXj2Qz 6dg0DiV0DrRfyk9L1cQGGjlcPd5oDcjlj0S/RN1byg0xZpQkCtxTUKqHOmjZkNMnX5Hb 2sv6j8gYsmRn/jpW1AJLuHmkoS8LFSQn/g+II9/mE0La8n8sccHxrLI2cnlW28I4oQJt 4WWcZwaSvVy4U076Wf3Hndt0HNRyBJ9NxW85XlZMMoQc5JxDcEtp3tGaHzwEX+0VxmRK /JkQ== X-Gm-Message-State: ACrzQf1Hbc0vXK1UxBvUe6+fhfdik1INqVncKb9/MnjGVTtDL4QoOxfA +8tKInRLT4u56jh2+J6TmW2a4h1WZl7UdFTmeA4CQPgZIJ29XYTe1s0FTihEpDKAfIM9hYXGDaQ QdEa9Z/zjqr6qxuLuyXrz3woFDej9A9iCp4hDxi/SkLj0gb1OII+RBZIOjDHCs98+ X-Received: by 2002:ad4:5bcd:0:b0:4ad:2fc4:6571 with SMTP id t13-20020ad45bcd000000b004ad2fc46571mr13633297qvt.103.1663714341343; Tue, 20 Sep 2022 15:52:21 -0700 (PDT) X-Google-Smtp-Source: AMsMyM49AII6Rns1H7ZmLUcEcjO26fJxeQLCFpCsmTaBJPdbSg0aeVmJr9Ni5Y1QCZMOUpeu1gIVgg== X-Received: by 2002:ad4:5bcd:0:b0:4ad:2fc4:6571 with SMTP id t13-20020ad45bcd000000b004ad2fc46571mr13633281qvt.103.1663714341100; Tue, 20 Sep 2022 15:52:21 -0700 (PDT) From: Peter Xu To: qemu-devel@nongnu.org Cc: peterx@redhat.com, Manish Mishra , Juan Quintela , ani@anisinha.ca, Leonardo Bras Soares Passos , "Dr . David Alan Gilbert" , "Daniel P . Berrange" Subject: [PATCH 09/14] migration: Add pss_init() Date: Tue, 20 Sep 2022 18:52:19 -0400 Message-Id: <20220920225219.48946-1-peterx@redhat.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20220920225106.48451-1-peterx@redhat.com> References: <20220920225106.48451-1-peterx@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=peterx@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -16 X-Spam_score: -1.7 X-Spam_bar: - X-Spam_report: (-1.7 / 5.0 requ) BAYES_00=-1.9, DKIM_INVALID=0.1, DKIM_SIGNED=0.1, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=no autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1663718770486100001 Content-Type: text/plain; charset="utf-8" Helper to init PSS structures. Signed-off-by: Peter Xu Reviewed-by: Dr. David Alan Gilbert --- migration/ram.c | 12 +++++++++--- 1 file changed, 9 insertions(+), 3 deletions(-) diff --git a/migration/ram.c b/migration/ram.c index 40ff5dc49f..b4b36ca59e 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -535,6 +535,14 @@ static bool do_compress_ram_page(QEMUFile *f, z_stream= *stream, RAMBlock *block, static void postcopy_preempt_restore(RAMState *rs, PageSearchStatus *pss, bool postcopy_requested); =20 +/* NOTE: page is the PFN not real ram_addr_t. */ +static void pss_init(PageSearchStatus *pss, RAMBlock *rb, ram_addr_t page) +{ + pss->block =3D rb; + pss->page =3D page; + pss->complete_round =3D false; +} + static void *do_data_compress(void *opaque) { CompressParam *param =3D opaque; @@ -2640,9 +2648,7 @@ static int ram_find_and_save_block(RAMState *rs) rs->last_page =3D 0; } =20 - pss.block =3D rs->last_seen_block; - pss.page =3D rs->last_page; - pss.complete_round =3D false; + pss_init(&pss, rs->last_seen_block, rs->last_page); =20 do { again =3D true; --=20 2.32.0 From nobody Fri Apr 26 23:13:09 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1663722044; cv=none; d=zohomail.com; s=zohoarc; b=gwpqGh8RYQ4EOr5LIEOymO02mNEMOuSXEuZd7QUbGDTuG3jjir3e50BwRM6TJWi83II8hg6smGEB/tEPppqlt1J+gmQF5Ka3ddNJuT2j9OwP8BfNZWtdchDa0zqeF3N3lx//9AjlC86Bi0pgTNuuMNNRvOgrq/DkEr4UBVki75c= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1663722044; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=leyo+AvhTJYUCn4Xc3eJcztvm2kX2JsxCJVxc3ElatE=; b=UG2T8IHrmIcqWGh3Dpe9Q2VS4hBJrMY3SXTO72Z8eKIPlhcq+rmey3fcuO8b51vfxSWqGJnZuaPdl+NzKuHAxzOv9qb6TaZyf8h5F6KftnHNI0bjy/e9HHCgWY6uLaTO5SSfsD8UNjoVlTp7M255a4bloEEcTfslVjh0IdP7rtM= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1663722044111465.23606982625984; Tue, 20 Sep 2022 18:00:44 -0700 (PDT) Received: from localhost ([::1]:56312 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1oao6S-000098-0U for importer@patchew.org; Tue, 20 Sep 2022 21:00:42 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:33450) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1oam6R-0005mI-Py for qemu-devel@nongnu.org; Tue, 20 Sep 2022 18:52:31 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:45834) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1oam6P-0001Uq-28 for qemu-devel@nongnu.org; Tue, 20 Sep 2022 18:52:30 -0400 Received: from mail-qk1-f200.google.com (mail-qk1-f200.google.com [209.85.222.200]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_128_GCM_SHA256) id us-mta-20-LnQjrf9SNx2snElSpWjKrQ-1; Tue, 20 Sep 2022 18:52:24 -0400 Received: by mail-qk1-f200.google.com with SMTP id bs33-20020a05620a472100b006cef8cfabe2so3085031qkb.12 for ; Tue, 20 Sep 2022 15:52:24 -0700 (PDT) Received: from localhost.localdomain (bras-base-aurron9127w-grc-46-70-31-27-79.dsl.bell.ca. [70.31.27.79]) by smtp.gmail.com with ESMTPSA id d18-20020ac851d2000000b00342fa1f4a10sm522191qtn.61.2022.09.20.15.52.22 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Tue, 20 Sep 2022 15:52:22 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1663714347; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=leyo+AvhTJYUCn4Xc3eJcztvm2kX2JsxCJVxc3ElatE=; b=WAaU4lRv8qhiZ3+ea44TecWYX0zJB6lmjpDZGZyiaBB2TDUJoKggZ1dD/exgHFl4Ne+BgE uKo8ukJtzude/PEakVSvnDL36Nqkqi4FGdsa9WquQm0SRw5VpPPhq/qnSCaAvLUqa4KavM 7mzLtHWMvTAv+RD97mSbPgAbBa9oxlk= X-MC-Unique: LnQjrf9SNx2snElSpWjKrQ-1 X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date; bh=leyo+AvhTJYUCn4Xc3eJcztvm2kX2JsxCJVxc3ElatE=; b=LiyfmcKDNaHXBPntSpb4LB6vn2v5CJWmjnEWhcSP2CfehUdn8D6/ZPTmi3/qA4LME0 YFVFeaXsejyICyaJ2QYKuILbgXXP30e8eZiTtbyfih3fVq2eqoUav+IL4ZiKmvcX61aU BvBnX0hTRTcnC/uQs0iIwzqXGfijH6LrXbO1lHWjyabnKn6eLNn/Y6Mer31FUKP9kZtM JAUwOvHUTQ+5HYsSV0FrvUpejC4nk/3a3oGvj04qZhjdC0qcSanBuv4vhuCBHLf8jwO/ ILj/9L4J8o2KP90ee9ASJPawrwaTCCgsfixOfDJitZ6stGzV6JTFK356Db3NIRsf8PdH GUBQ== X-Gm-Message-State: ACrzQf0OVl1X29ObqandmxgYbIwDX3c8ibRDT6gabbcKHZAet1JBQTPn xRSG7k/0GUM9llySq00nOkymV06rqxWGt/KqVtkAWE+eGM7dAUgWjZuMLs8Lwekrtt0816n8bEM vtR3HK2OYlTPqPG3BEw7gEJU9zDXpGMWD6gwh7VYabSGWPxFfg+d3GY/QjABtbvTc X-Received: by 2002:ad4:5bc5:0:b0:4ad:34db:1f0f with SMTP id t5-20020ad45bc5000000b004ad34db1f0fmr11745174qvt.106.1663714343501; Tue, 20 Sep 2022 15:52:23 -0700 (PDT) X-Google-Smtp-Source: AMsMyM7ob+lGuKb5WNqUbKXgy3E5wEGhT/A+kkVx4NNZb0a9zF1SVpJmgkx8u0Y6rYjFfgb/RVSGNQ== X-Received: by 2002:ad4:5bc5:0:b0:4ad:34db:1f0f with SMTP id t5-20020ad45bc5000000b004ad34db1f0fmr11745147qvt.106.1663714343184; Tue, 20 Sep 2022 15:52:23 -0700 (PDT) From: Peter Xu To: qemu-devel@nongnu.org Cc: peterx@redhat.com, Manish Mishra , Juan Quintela , ani@anisinha.ca, Leonardo Bras Soares Passos , "Dr . David Alan Gilbert" , "Daniel P . Berrange" Subject: [PATCH 10/14] migration: Make PageSearchStatus part of RAMState Date: Tue, 20 Sep 2022 18:52:21 -0400 Message-Id: <20220920225221.48999-1-peterx@redhat.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20220920225106.48451-1-peterx@redhat.com> References: <20220920225106.48451-1-peterx@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=peterx@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1663722045575100001 Content-Type: text/plain; charset="utf-8" We used to allocate PSS structure on the stack for precopy when sending pages. Make it static, so as to describe per-channel ram migration status. Here we declared RAM_CHANNEL_MAX instances, preparing for postcopy to use it, even though this patch has not yet to start using the 2nd instance. This should not have any functional change per se, but it already starts to export PSS information via the RAMState, so that e.g. one PSS channel can start to reference the other PSS channel. Always protect PSS access using the same RAMState.bitmap_mutex. We already do so, so no code change needed, just some comment update. Maybe we should consider renaming bitmap_mutex some day as it's going to be a more commonly and big mutex we use for ram states, but just leave it for later. Signed-off-by: Peter Xu Reviewed-by: Dr. David Alan Gilbert --- migration/ram.c | 112 ++++++++++++++++++++++++++---------------------- 1 file changed, 61 insertions(+), 51 deletions(-) diff --git a/migration/ram.c b/migration/ram.c index b4b36ca59e..dbe11e1ace 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -85,6 +85,46 @@ =20 XBZRLECacheStats xbzrle_counters; =20 +/* used by the search for pages to send */ +struct PageSearchStatus { + /* The migration channel used for a specific host page */ + QEMUFile *pss_channel; + /* Current block being searched */ + RAMBlock *block; + /* Current page to search from */ + unsigned long page; + /* Set once we wrap around */ + bool complete_round; + /* + * [POSTCOPY-ONLY] Whether current page is explicitly requested by + * postcopy. When set, the request is "urgent" because the dest QEMU + * threads are waiting for us. + */ + bool postcopy_requested; + /* + * [POSTCOPY-ONLY] The target channel to use to send current page. + * + * Note: This may _not_ match with the value in postcopy_requested + * above. Let's imagine the case where the postcopy request is exactly + * the page that we're sending in progress during precopy. In this case + * we'll have postcopy_requested set to true but the target channel + * will be the precopy channel (so that we don't split brain on that + * specific page since the precopy channel already contains partial of + * that page data). + * + * Besides that specific use case, postcopy_target_channel should + * always be equal to postcopy_requested, because by default we send + * postcopy pages via postcopy preempt channel. + */ + bool postcopy_target_channel; + /* Whether we're sending a host page */ + bool host_page_sending; + /* The start/end of current host page. Invalid if host_page_sending= =3D=3Dfalse */ + unsigned long host_page_start; + unsigned long host_page_end; +}; +typedef struct PageSearchStatus PageSearchStatus; + /* struct contains XBZRLE cache and a static page used by the compression */ static struct { @@ -319,6 +359,11 @@ typedef struct { struct RAMState { /* QEMUFile used for this migration */ QEMUFile *f; + /* + * PageSearchStatus structures for the channels when send pages. + * Protected by the bitmap_mutex. + */ + PageSearchStatus pss[RAM_CHANNEL_MAX]; /* UFFD file descriptor, used in 'write-tracking' migration */ int uffdio_fd; /* Last block that we have visited searching for dirty pages */ @@ -362,7 +407,12 @@ struct RAMState { uint64_t target_page_count; /* number of dirty bits in the bitmap */ uint64_t migration_dirty_pages; - /* Protects modification of the bitmap and migration dirty pages */ + /* + * Protects: + * - dirty/clear bitmap + * - migration_dirty_pages + * - pss structures + */ QemuMutex bitmap_mutex; /* The RAMBlock used in the last src_page_requests */ RAMBlock *last_req_rb; @@ -444,46 +494,6 @@ void dirty_sync_missed_zero_copy(void) ram_counters.dirty_sync_missed_zero_copy++; } =20 -/* used by the search for pages to send */ -struct PageSearchStatus { - /* The migration channel used for a specific host page */ - QEMUFile *pss_channel; - /* Current block being searched */ - RAMBlock *block; - /* Current page to search from */ - unsigned long page; - /* Set once we wrap around */ - bool complete_round; - /* - * [POSTCOPY-ONLY] Whether current page is explicitly requested by - * postcopy. When set, the request is "urgent" because the dest QEMU - * threads are waiting for us. - */ - bool postcopy_requested; - /* - * [POSTCOPY-ONLY] The target channel to use to send current page. - * - * Note: This may _not_ match with the value in postcopy_requested - * above. Let's imagine the case where the postcopy request is exactly - * the page that we're sending in progress during precopy. In this case - * we'll have postcopy_requested set to true but the target channel - * will be the precopy channel (so that we don't split brain on that - * specific page since the precopy channel already contains partial of - * that page data). - * - * Besides that specific use case, postcopy_target_channel should - * always be equal to postcopy_requested, because by default we send - * postcopy pages via postcopy preempt channel. - */ - bool postcopy_target_channel; - /* Whether we're sending a host page */ - bool host_page_sending; - /* The start/end of current host page. Only valid if host_page_sendin= g=3D=3Dtrue */ - unsigned long host_page_start; - unsigned long host_page_end; -}; -typedef struct PageSearchStatus PageSearchStatus; - CompressionStats compression_counters; =20 struct CompressParam { @@ -2627,7 +2637,7 @@ static int ram_save_host_page(RAMState *rs, PageSearc= hStatus *pss) */ static int ram_find_and_save_block(RAMState *rs) { - PageSearchStatus pss; + PageSearchStatus *pss =3D &rs->pss[RAM_CHANNEL_PRECOPY]; int pages =3D 0; bool again, found; =20 @@ -2648,11 +2658,11 @@ static int ram_find_and_save_block(RAMState *rs) rs->last_page =3D 0; } =20 - pss_init(&pss, rs->last_seen_block, rs->last_page); + pss_init(pss, rs->last_seen_block, rs->last_page); =20 do { again =3D true; - found =3D get_queued_page(rs, &pss); + found =3D get_queued_page(rs, pss); =20 if (!found) { /* @@ -2660,27 +2670,27 @@ static int ram_find_and_save_block(RAMState *rs) * preempted precopy. Otherwise find the next dirty bit. */ if (postcopy_preempt_triggered(rs)) { - postcopy_preempt_restore(rs, &pss, false); + postcopy_preempt_restore(rs, pss, false); found =3D true; } else { /* priority queue empty, so just search for something dirt= y */ - found =3D find_dirty_block(rs, &pss, &again); + found =3D find_dirty_block(rs, pss, &again); } } =20 if (found) { /* Update rs->f with correct channel */ if (postcopy_preempt_active()) { - postcopy_preempt_choose_channel(rs, &pss); + postcopy_preempt_choose_channel(rs, pss); } /* Cache rs->f in pss_channel (TODO: remove rs->f) */ - pss.pss_channel =3D rs->f; - pages =3D ram_save_host_page(rs, &pss); + pss->pss_channel =3D rs->f; + pages =3D ram_save_host_page(rs, pss); } } while (!pages && again); =20 - rs->last_seen_block =3D pss.block; - rs->last_page =3D pss.page; + rs->last_seen_block =3D pss->block; + rs->last_page =3D pss->page; =20 return pages; } --=20 2.32.0 From nobody Fri Apr 26 23:13:09 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1663719671; cv=none; d=zohomail.com; s=zohoarc; b=e90Xp7gKB5jpJqUmy68K6b/fsO+JTY+2A2uz1yWoSHkCuArOmyI/vtK5NEUuddxHsHnNxH2+WSzAOprKnpM6lSF+mCYfIuYlLeJSCwSJjVJSc/HhA43A4Mcsgg8VevkgZnXPjDj/8yNtb8n+6IxM/Kg1yJ6WLh6WwFbail/4WP8= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1663719671; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=xjyxsHXLlqCkXQX3xVM0s9I2tYX2rFDZ/QNGQ/MXgNI=; b=O1BcOUJ0kqzzCSbNuqK/5wGzqkIY91HTXSCNnEcLl7gn/xSjSRN3ZZE7G1NFgJ6exKz/chdz8LnzKMD+Y+OAWUBb5IJOZqhSx7uY9AwGZdRLaGDv7pGwJJFS5x/uts5xjUqcYOhQGhbCH0AAcacAlovAkq7ToZZlYjoADYeHeCI= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1663719671944492.6167172916521; Tue, 20 Sep 2022 17:21:11 -0700 (PDT) Received: from localhost ([::1]:43344 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1oanUE-0004wp-Sj for importer@patchew.org; Tue, 20 Sep 2022 20:21:10 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:33452) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1oam6R-0005nH-9J for qemu-devel@nongnu.org; Tue, 20 Sep 2022 18:52:31 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]:39936) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1oam6P-0001Us-2l for qemu-devel@nongnu.org; Tue, 20 Sep 2022 18:52:30 -0400 Received: from mail-qt1-f197.google.com (mail-qt1-f197.google.com [209.85.160.197]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_128_GCM_SHA256) id us-mta-635-1I2nRThtMGum9IpQI97SeA-1; Tue, 20 Sep 2022 18:52:26 -0400 Received: by mail-qt1-f197.google.com with SMTP id fv21-20020a05622a4a1500b0035cc9b4fc20so2892352qtb.12 for ; Tue, 20 Sep 2022 15:52:26 -0700 (PDT) Received: from localhost.localdomain (bras-base-aurron9127w-grc-46-70-31-27-79.dsl.bell.ca. [70.31.27.79]) by smtp.gmail.com with ESMTPSA id n18-20020a05622a041200b0035a71a41645sm637377qtx.37.2022.09.20.15.52.24 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Tue, 20 Sep 2022 15:52:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1663714347; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=xjyxsHXLlqCkXQX3xVM0s9I2tYX2rFDZ/QNGQ/MXgNI=; b=fgPlLMazzD9r7rt35wihg6JtIO1Kc0O1UpxjU+f9kSWef9HMzLWUv2zi7GblAgNOesXc6d Cx+XbDO3bMJWb/ylxGwsInmJq6J1UrHJuBz8TnRCo3Jw+uwjSS9KvVUyYyNA3IjiMaKWZx D019FTYTuXsY57MQK5gHT2SNP/0WlAw= X-MC-Unique: 1I2nRThtMGum9IpQI97SeA-1 X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date; bh=xjyxsHXLlqCkXQX3xVM0s9I2tYX2rFDZ/QNGQ/MXgNI=; b=Tmed2CSKyVhBCL7VYjPwRr+mI2RT+ympLU3pD3bSarSK7tEOhbEJRoy0eqTPg+v/b0 m1mpezmc0L5ncITxjRQq+gX4wY9cn5SK83Wuc1DJfnvmtu371GRCLdiWgQmFDyC40tvF djDOWhtkMDOcVJ8bGCcXtt+OHT8nQoOXrjEjTNB3r77RBEysuFueoklndDG+xHmYbZaM QJYja1YVn/eqqm3aE1oVMWngjiC5U9kO3q7vfq39cgvli+16jMm0IG0k/Xl2ufEMLTum Aj05p6h0dcxfyYvOR6GARlmN+DxQTS65Pg0veBW0RXJdBPKJsDDvMJTCOWU1FzwMjAOG I8KA== X-Gm-Message-State: ACrzQf0XmM+ZkgOMFTpn6O5hJ7KwOB700vU32txSZJjxpBXYqXP9NUUf p21D2IKXntRRIW3OkgkzvwNj3SmrrbpGVmVWUheYF3OqyzZDFltMrHBu/Or45pIwXvf8tb6k+cM WLGRoFkii1ZYr4MDgUqNiFCbXd8BNxI7ccxN2+lkM0huXjOsU6F25sQ+AU69guwze X-Received: by 2002:a05:622a:1790:b0:35c:8450:d9e4 with SMTP id s16-20020a05622a179000b0035c8450d9e4mr21269346qtk.130.1663714345610; Tue, 20 Sep 2022 15:52:25 -0700 (PDT) X-Google-Smtp-Source: AMsMyM649vcv9JEQczX/NjT/lxzB7bGFNYZtLjXqtO1LbIwffkQEYSbpnueEa76bR5jr+WVUqYsv9Q== X-Received: by 2002:a05:622a:1790:b0:35c:8450:d9e4 with SMTP id s16-20020a05622a179000b0035c8450d9e4mr21269318qtk.130.1663714345250; Tue, 20 Sep 2022 15:52:25 -0700 (PDT) From: Peter Xu To: qemu-devel@nongnu.org Cc: peterx@redhat.com, Manish Mishra , Juan Quintela , ani@anisinha.ca, Leonardo Bras Soares Passos , "Dr . David Alan Gilbert" , "Daniel P . Berrange" Subject: [PATCH 11/14] migration: Move last_sent_block into PageSearchStatus Date: Tue, 20 Sep 2022 18:52:23 -0400 Message-Id: <20220920225223.49052-1-peterx@redhat.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20220920225106.48451-1-peterx@redhat.com> References: <20220920225106.48451-1-peterx@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=peterx@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1663719672928100001 Content-Type: text/plain; charset="utf-8" Since we use PageSearchStatus to represent a channel, it makes perfect sense to keep last_sent_block (aka, leverage RAM_SAVE_FLAG_CONTINUE) to be per-channel rather than global because each channel can be sending different pages on ramblocks. Hence move it from RAMState into PageSearchStatus. Signed-off-by: Peter Xu Reviewed-by: Dr. David Alan Gilbert --- migration/ram.c | 71 ++++++++++++++++++++++++++++--------------------- 1 file changed, 41 insertions(+), 30 deletions(-) diff --git a/migration/ram.c b/migration/ram.c index dbe11e1ace..fdcb61a2c8 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -89,6 +89,8 @@ XBZRLECacheStats xbzrle_counters; struct PageSearchStatus { /* The migration channel used for a specific host page */ QEMUFile *pss_channel; + /* Last block from where we have sent data */ + RAMBlock *last_sent_block; /* Current block being searched */ RAMBlock *block; /* Current page to search from */ @@ -368,8 +370,6 @@ struct RAMState { int uffdio_fd; /* Last block that we have visited searching for dirty pages */ RAMBlock *last_seen_block; - /* Last block from where we have sent data */ - RAMBlock *last_sent_block; /* Last dirty target page we have sent */ ram_addr_t last_page; /* last ram version we have seen */ @@ -677,16 +677,17 @@ exit: * * Returns the number of bytes written * - * @f: QEMUFile where to send the data + * @pss: current PSS channel status * @block: block that contains the page we want to send * @offset: offset inside the block for the page * in the lower bits, it contains flags */ -static size_t save_page_header(RAMState *rs, QEMUFile *f, RAMBlock *block, +static size_t save_page_header(PageSearchStatus *pss, RAMBlock *block, ram_addr_t offset) { size_t size, len; - bool same_block =3D (block =3D=3D rs->last_sent_block); + bool same_block =3D (block =3D=3D pss->last_sent_block); + QEMUFile *f =3D pss->pss_channel; =20 if (same_block) { offset |=3D RAM_SAVE_FLAG_CONTINUE; @@ -699,7 +700,7 @@ static size_t save_page_header(RAMState *rs, QEMUFile *= f, RAMBlock *block, qemu_put_byte(f, len); qemu_put_buffer(f, (uint8_t *)block->idstr, len); size +=3D 1 + len; - rs->last_sent_block =3D block; + pss->last_sent_block =3D block; } return size; } @@ -783,17 +784,19 @@ static void xbzrle_cache_zero_page(RAMState *rs, ram_= addr_t current_addr) * -1 means that xbzrle would be longer than normal * * @rs: current RAM state + * @pss: current PSS channel * @current_data: pointer to the address of the page contents * @current_addr: addr of the page * @block: block that contains the page we want to send * @offset: offset inside the block for the page */ -static int save_xbzrle_page(RAMState *rs, QEMUFile *file, +static int save_xbzrle_page(RAMState *rs, PageSearchStatus *pss, uint8_t **current_data, ram_addr_t current_add= r, RAMBlock *block, ram_addr_t offset) { int encoded_len =3D 0, bytes_xbzrle; uint8_t *prev_cached_page; + QEMUFile *file =3D pss->pss_channel; =20 if (!cache_is_cached(XBZRLE.cache, current_addr, ram_counters.dirty_sync_count)) { @@ -858,7 +861,7 @@ static int save_xbzrle_page(RAMState *rs, QEMUFile *fil= e, } =20 /* Send XBZRLE based compressed page */ - bytes_xbzrle =3D save_page_header(rs, file, block, + bytes_xbzrle =3D save_page_header(pss, block, offset | RAM_SAVE_FLAG_XBZRLE); qemu_put_byte(file, ENCODING_FLAG_XBZRLE); qemu_put_be16(file, encoded_len); @@ -1289,19 +1292,19 @@ static void ram_release_page(const char *rbname, ui= nt64_t offset) * Returns the size of data written to the file, 0 means the page is not * a zero page * - * @rs: current RAM state - * @file: the file where the data is saved + * @pss: current PSS channel * @block: block that contains the page we want to send * @offset: offset inside the block for the page */ -static int save_zero_page_to_file(RAMState *rs, QEMUFile *file, +static int save_zero_page_to_file(PageSearchStatus *pss, RAMBlock *block, ram_addr_t offset) { uint8_t *p =3D block->host + offset; + QEMUFile *file =3D pss->pss_channel; int len =3D 0; =20 if (buffer_is_zero(p, TARGET_PAGE_SIZE)) { - len +=3D save_page_header(rs, file, block, offset | RAM_SAVE_FLAG_= ZERO); + len +=3D save_page_header(pss, block, offset | RAM_SAVE_FLAG_ZERO); qemu_put_byte(file, 0); len +=3D 1; ram_release_page(block->idstr, offset); @@ -1314,14 +1317,14 @@ static int save_zero_page_to_file(RAMState *rs, QEM= UFile *file, * * Returns the number of pages written. * - * @rs: current RAM state + * @pss: current PSS channel * @block: block that contains the page we want to send * @offset: offset inside the block for the page */ -static int save_zero_page(RAMState *rs, QEMUFile *file, RAMBlock *block, +static int save_zero_page(PageSearchStatus *pss, RAMBlock *block, ram_addr_t offset) { - int len =3D save_zero_page_to_file(rs, file, block, offset); + int len =3D save_zero_page_to_file(pss, block, offset); =20 if (len) { qatomic_inc(&ram_counters.duplicate); @@ -1374,16 +1377,18 @@ static bool control_save_page(PageSearchStatus *pss= , RAMBlock *block, * * Returns the number of pages written. * - * @rs: current RAM state + * @pss: current PSS channel * @block: block that contains the page we want to send * @offset: offset inside the block for the page * @buf: the page to be sent * @async: send to page asyncly */ -static int save_normal_page(RAMState *rs, QEMUFile *file, RAMBlock *block, +static int save_normal_page(PageSearchStatus *pss, RAMBlock *block, ram_addr_t offset, uint8_t *buf, bool async) { - ram_transferred_add(save_page_header(rs, file, block, + QEMUFile *file =3D pss->pss_channel; + + ram_transferred_add(save_page_header(pss, block, offset | RAM_SAVE_FLAG_PAGE)); if (async) { qemu_put_buffer_async(file, buf, TARGET_PAGE_SIZE, @@ -1423,7 +1428,7 @@ static int ram_save_page(RAMState *rs, PageSearchStat= us *pss) =20 XBZRLE_cache_lock(); if (rs->xbzrle_enabled && !migration_in_postcopy()) { - pages =3D save_xbzrle_page(rs, pss->pss_channel, &p, current_addr, + pages =3D save_xbzrle_page(rs, pss, &p, current_addr, block, offset); if (!rs->last_stage) { /* Can't send this cached data async, since the cache page @@ -1435,8 +1440,7 @@ static int ram_save_page(RAMState *rs, PageSearchStat= us *pss) =20 /* XBZRLE overflow or normal page */ if (pages =3D=3D -1) { - pages =3D save_normal_page(rs, pss->pss_channel, block, offset, - p, send_async); + pages =3D save_normal_page(pss, block, offset, p, send_async); } =20 XBZRLE_cache_unlock(); @@ -1459,14 +1463,15 @@ static bool do_compress_ram_page(QEMUFile *f, z_str= eam *stream, RAMBlock *block, ram_addr_t offset, uint8_t *source_buf) { RAMState *rs =3D ram_state; + PageSearchStatus *pss =3D &rs->pss[RAM_CHANNEL_PRECOPY]; uint8_t *p =3D block->host + offset; int ret; =20 - if (save_zero_page_to_file(rs, f, block, offset)) { + if (save_zero_page_to_file(pss, block, offset)) { return true; } =20 - save_page_header(rs, f, block, offset | RAM_SAVE_FLAG_COMPRESS_PAGE); + save_page_header(pss, block, offset | RAM_SAVE_FLAG_COMPRESS_PAGE); =20 /* * copy it to a internal buffer to avoid it being modified by VM @@ -2286,7 +2291,8 @@ static bool save_page_use_compression(RAMState *rs) * has been properly handled by compression, otherwise needs other * paths to handle it */ -static bool save_compress_page(RAMState *rs, RAMBlock *block, ram_addr_t o= ffset) +static bool save_compress_page(RAMState *rs, PageSearchStatus *pss, + RAMBlock *block, ram_addr_t offset) { if (!save_page_use_compression(rs)) { return false; @@ -2302,7 +2308,7 @@ static bool save_compress_page(RAMState *rs, RAMBlock= *block, ram_addr_t offset) * We post the fist page as normal page as compression will take * much CPU resource. */ - if (block !=3D rs->last_sent_block) { + if (block !=3D pss->last_sent_block) { flush_compressed_data(rs); return false; } @@ -2333,11 +2339,11 @@ static int ram_save_target_page(RAMState *rs, PageS= earchStatus *pss) return res; } =20 - if (save_compress_page(rs, block, offset)) { + if (save_compress_page(rs, pss, block, offset)) { return 1; } =20 - res =3D save_zero_page(rs, pss->pss_channel, block, offset); + res =3D save_zero_page(pss, block, offset); if (res > 0) { /* Must let xbzrle know, otherwise a previous (now 0'd) cached * page would be stale @@ -2469,7 +2475,7 @@ static void postcopy_preempt_choose_channel(RAMState = *rs, PageSearchStatus *pss) * If channel switched, reset last_sent_block since the old sent b= lock * may not be on the same channel. */ - rs->last_sent_block =3D NULL; + pss->last_sent_block =3D NULL; =20 trace_postcopy_preempt_switch_channel(channel); } @@ -2804,8 +2810,13 @@ static void ram_save_cleanup(void *opaque) =20 static void ram_state_reset(RAMState *rs) { + int i; + + for (i =3D 0; i < RAM_CHANNEL_MAX; i++) { + rs->pss[i].last_sent_block =3D NULL; + } + rs->last_seen_block =3D NULL; - rs->last_sent_block =3D NULL; rs->last_page =3D 0; rs->last_version =3D ram_list.version; rs->xbzrle_enabled =3D false; @@ -2999,8 +3010,8 @@ void ram_postcopy_send_discard_bitmap(MigrationState = *ms) migration_bitmap_sync(rs); =20 /* Easiest way to make sure we don't resume in the middle of a host-pa= ge */ + rs->pss[RAM_CHANNEL_PRECOPY].last_sent_block =3D NULL; rs->last_seen_block =3D NULL; - rs->last_sent_block =3D NULL; rs->last_page =3D 0; =20 postcopy_each_ram_send_discard(ms); --=20 2.32.0 From nobody Fri Apr 26 23:13:09 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1663719875; cv=none; d=zohomail.com; s=zohoarc; b=j7W1RvjbTD2qft0XqIsciidQmiE5JX78I2odjYvTfEe3ZJQ17IUO4aUjg+5nMD+0CCaIRVSDSU7JHOZNGvRdfGHGa7NbgAtVSd4In2/DcBRRL1ysFyZXdw8GKSWoFN4kWk77lOdxFG7wQNzQ5KWa2BNDZtTfBpDSyhTnSDPivtU= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1663719875; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=vlIFQOT3gtE++aswKM6B91kX2+TR/FlMeT6k15H9zVQ=; b=b/+ANj45jno7rXsTtVjx6u4GhtvEYocZbamTVT4ZEGiWI2a+ViyH3ft2ZuAn2fmWXsl8ziX4GvM3UuYfKI16w/NCTv/LrCtaVYUwYyBYvobSqtvsrNByijhwd7L3Ez8aLVFUXTNUFLVj9SDOuXWj7tNoNDUIWhJgqKOokbMZK6o= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1663719875076608.6237269550126; Tue, 20 Sep 2022 17:24:35 -0700 (PDT) Received: from localhost ([::1]:44062 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1oanXU-0002aA-O8 for importer@patchew.org; Tue, 20 Sep 2022 20:24:32 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:33454) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1oam6S-0005sY-OA for qemu-devel@nongnu.org; Tue, 20 Sep 2022 18:52:32 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]:53933) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1oam6Q-0001V0-8S for qemu-devel@nongnu.org; Tue, 20 Sep 2022 18:52:32 -0400 Received: from mail-qk1-f200.google.com (mail-qk1-f200.google.com [209.85.222.200]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_128_GCM_SHA256) id us-mta-629-6x8tiKStMJ2O0wcmRBdnTg-1; Tue, 20 Sep 2022 18:52:28 -0400 Received: by mail-qk1-f200.google.com with SMTP id d18-20020a05620a241200b006ce80a4d74aso3090618qkn.6 for ; Tue, 20 Sep 2022 15:52:28 -0700 (PDT) Received: from localhost.localdomain (bras-base-aurron9127w-grc-46-70-31-27-79.dsl.bell.ca. [70.31.27.79]) by smtp.gmail.com with ESMTPSA id v20-20020a05620a0f1400b006bb87c4833asm822034qkl.109.2022.09.20.15.52.26 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Tue, 20 Sep 2022 15:52:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1663714349; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=vlIFQOT3gtE++aswKM6B91kX2+TR/FlMeT6k15H9zVQ=; b=SIirHhIQo3LcukbXeokAj58x81bdjm3qWVy62gQ83TQmDsMXDuAiR81aZ6fslA7yWpLLmu XsN2pIuesJQvskqUhjZMjC+9Nn+EK9iaVKkVlHkwBt871eShFFY/sCys8G4yukNKmZNY6E gz1JUL/NbFzYerHGjC777ZItJC1GVYM= X-MC-Unique: 6x8tiKStMJ2O0wcmRBdnTg-1 X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date; bh=vlIFQOT3gtE++aswKM6B91kX2+TR/FlMeT6k15H9zVQ=; b=Xjd24ik7XMiTXAHu0cjXuEzOstG8D2f0iqNLG05q+eVDinNtOkpQenh8OwdvCfYoea GsadROm5q+3SXVsH2fDE2lvzdPsLe5PYhPRO3dLt3EM7Zd7P4qB4X0PDwBphLjownWYv nVFEzOXx0W8a7deyFMADYwB+dAxUDRJs/5BgWslJgSqiMhdhtvDyqjV/LTDM06wxeUmt XghjNjMo+XWMdWFos3ilBXiL7QmsSt3PUX2WuXrboSkUMbDH1goEe9haAOrNMCO1rgwQ nAhQXCp3cvyzVfm1Xuiek48lZ2Z8LlYt8lMdIYYGS5+0A7wbXwbDXyeET8r4wMiVXEzJ C1dQ== X-Gm-Message-State: ACrzQf3CUQ2dppPvJLiY0dHyFdKr8L0mGIXkdoc4C4whYqJjA6l75hPO L6z1oMq+Yqz5ohkJoEeUK5a26bYDaNmndkMU55PnK8fyQ4gb+dTUvKai8qeVjGm0xj7S/4w7ucv XHVHcV46OsAbKXGK9eucYM5nYHo8NbpnjeDfQo0+K5zTIurK3P687yCS04qEFJpit X-Received: by 2002:ac8:5e54:0:b0:35c:c7f1:7234 with SMTP id i20-20020ac85e54000000b0035cc7f17234mr21682375qtx.656.1663714347903; Tue, 20 Sep 2022 15:52:27 -0700 (PDT) X-Google-Smtp-Source: AMsMyM5uSknqK8PGZu/bJ37qHlBKityhkglhvDnP7ouethzMZsz2hQgIbu7STdQZxKTdkwdusuabVQ== X-Received: by 2002:ac8:5e54:0:b0:35c:c7f1:7234 with SMTP id i20-20020ac85e54000000b0035cc7f17234mr21682359qtx.656.1663714347545; Tue, 20 Sep 2022 15:52:27 -0700 (PDT) From: Peter Xu To: qemu-devel@nongnu.org Cc: peterx@redhat.com, Manish Mishra , Juan Quintela , ani@anisinha.ca, Leonardo Bras Soares Passos , "Dr . David Alan Gilbert" , "Daniel P . Berrange" Subject: [PATCH 12/14] migration: Send requested page directly in rp-return thread Date: Tue, 20 Sep 2022 18:52:25 -0400 Message-Id: <20220920225225.49105-1-peterx@redhat.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20220920225106.48451-1-peterx@redhat.com> References: <20220920225106.48451-1-peterx@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=peterx@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -21 X-Spam_score: -2.2 X-Spam_bar: -- X-Spam_report: (-2.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URG_BIZ=0.573 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1663719875558100001 Content-Type: text/plain; charset="utf-8" With all the facilities ready, send the requested page directly in the rp-return thread rather than queuing it in the request queue, if and only if postcopy preempt is enabled. It can achieve so because it uses separate channel for sending urgent pages. The only shared data is bitmap and it's protected by the bitmap_mutex. Note that since we're moving the ownership of the urgent channel from the migration thread to rp thread it also means the rp thread is responsible for managing the qemufile, e.g. properly close it when pausing migration happens. For this, let migration_release_from_dst_file to cover shutdown of the urgent channel too, renaming it as migration_release_dst_files() to better show what it does. Signed-off-by: Peter Xu Reviewed-by: Dr. David Alan Gilbert --- migration/migration.c | 35 +++++++------ migration/ram.c | 112 ++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 131 insertions(+), 16 deletions(-) diff --git a/migration/migration.c b/migration/migration.c index 0eacc0c99b..fae8fd378b 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -2845,8 +2845,11 @@ static int migrate_handle_rp_resume_ack(MigrationSta= te *s, uint32_t value) return 0; } =20 -/* Release ms->rp_state.from_dst_file in a safe way */ -static void migration_release_from_dst_file(MigrationState *ms) +/* + * Release ms->rp_state.from_dst_file (and postcopy_qemufile_src if + * existed) in a safe way. + */ +static void migration_release_dst_files(MigrationState *ms) { QEMUFile *file; =20 @@ -2859,6 +2862,18 @@ static void migration_release_from_dst_file(Migratio= nState *ms) ms->rp_state.from_dst_file =3D NULL; } =20 + /* + * Do the same to postcopy fast path socket too if there is. No + * locking needed because this qemufile should only be managed by + * return path thread. + */ + if (ms->postcopy_qemufile_src) { + migration_ioc_unregister_yank_from_file(ms->postcopy_qemufile_src); + qemu_file_shutdown(ms->postcopy_qemufile_src); + qemu_fclose(ms->postcopy_qemufile_src); + ms->postcopy_qemufile_src =3D NULL; + } + qemu_fclose(file); } =20 @@ -3003,7 +3018,7 @@ out: * Maybe there is something we can do: it looks like a * network down issue, and we pause for a recovery. */ - migration_release_from_dst_file(ms); + migration_release_dst_files(ms); rp =3D NULL; if (postcopy_pause_return_path_thread(ms)) { /* @@ -3021,7 +3036,7 @@ out: } =20 trace_source_return_path_thread_end(); - migration_release_from_dst_file(ms); + migration_release_dst_files(ms); rcu_unregister_thread(); return NULL; } @@ -3544,18 +3559,6 @@ static MigThrError postcopy_pause(MigrationState *s) qemu_file_shutdown(file); qemu_fclose(file); =20 - /* - * Do the same to postcopy fast path socket too if there is. No - * locking needed because no racer as long as we do this before se= tting - * status to paused. - */ - if (s->postcopy_qemufile_src) { - migration_ioc_unregister_yank_from_file(s->postcopy_qemufile_s= rc); - qemu_file_shutdown(s->postcopy_qemufile_src); - qemu_fclose(s->postcopy_qemufile_src); - s->postcopy_qemufile_src =3D NULL; - } - migrate_set_state(&s->state, s->state, MIGRATION_STATUS_POSTCOPY_PAUSED); =20 diff --git a/migration/ram.c b/migration/ram.c index fdcb61a2c8..fd301d793c 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -539,6 +539,8 @@ static QemuThread *decompress_threads; static QemuMutex decomp_done_lock; static QemuCond decomp_done_cond; =20 +static int ram_save_host_page_urgent(PageSearchStatus *pss); + static bool do_compress_ram_page(QEMUFile *f, z_stream *stream, RAMBlock *= block, ram_addr_t offset, uint8_t *source_buf); =20 @@ -553,6 +555,16 @@ static void pss_init(PageSearchStatus *pss, RAMBlock *= rb, ram_addr_t page) pss->complete_round =3D false; } =20 +/* + * Check whether two PSSs are actively sending the same page. Return true + * if it is, false otherwise. + */ +static bool pss_overlap(PageSearchStatus *pss1, PageSearchStatus *pss2) +{ + return pss1->host_page_sending && pss2->host_page_sending && + (pss1->host_page_start =3D=3D pss2->host_page_start); +} + static void *do_data_compress(void *opaque) { CompressParam *param =3D opaque; @@ -2253,6 +2265,57 @@ int ram_save_queue_pages(const char *rbname, ram_add= r_t start, ram_addr_t len) return -1; } =20 + /* + * When with postcopy preempt, we send back the page directly in the + * rp-return thread. + */ + if (postcopy_preempt_active()) { + ram_addr_t page_start =3D start >> TARGET_PAGE_BITS; + size_t page_size =3D qemu_ram_pagesize(ramblock); + PageSearchStatus *pss =3D &ram_state->pss[RAM_CHANNEL_POSTCOPY]; + int ret =3D 0; + + qemu_mutex_lock(&rs->bitmap_mutex); + + pss_init(pss, ramblock, page_start); + /* + * Always use the preempt channel, and make sure it's there. It's + * safe to access without lock, because when rp-thread is running + * we should be the only one who operates on the qemufile + */ + pss->pss_channel =3D migrate_get_current()->postcopy_qemufile_src; + pss->postcopy_requested =3D true; + assert(pss->pss_channel); + + /* + * It must be either one or multiple of host page size. Just + * assert; if something wrong we're mostly split brain anyway. + */ + assert(len % page_size =3D=3D 0); + while (len) { + if (ram_save_host_page_urgent(pss)) { + error_report("%s: ram_save_host_page_urgent() failed: " + "ramblock=3D%s, start_addr=3D0x"RAM_ADDR_FMT, + __func__, ramblock->idstr, start); + ret =3D -1; + break; + } + /* + * NOTE: after ram_save_host_page_urgent() succeeded, pss->page + * will automatically be moved and point to the next host page + * we're going to send, so no need to update here. + * + * Normally QEMU never sends >1 host page in requests, so + * logically we don't even need that as the loop should only + * run once, but just to be consistent. + */ + len -=3D page_size; + }; + qemu_mutex_unlock(&rs->bitmap_mutex); + + return ret; + } + struct RAMSrcPageRequest *new_entry =3D g_new0(struct RAMSrcPageRequest, 1); new_entry->rb =3D ramblock; @@ -2531,6 +2594,55 @@ static void pss_host_page_finish(PageSearchStatus *p= ss) pss->host_page_start =3D pss->host_page_end =3D 0; } =20 +/* + * Send an urgent host page specified by `pss'. Need to be called with + * bitmap_mutex held. + * + * Returns 0 if save host page succeeded, false otherwise. + */ +static int ram_save_host_page_urgent(PageSearchStatus *pss) +{ + bool page_dirty, sent =3D false; + RAMState *rs =3D ram_state; + int ret =3D 0; + + trace_postcopy_preempt_send_host_page(pss->block->idstr, pss->page); + pss_host_page_prepare(pss); + + /* + * If precopy is sending the same page, let it be done in precopy, or + * we could send the same page in two channels and none of them will + * receive the whole page. + */ + if (pss_overlap(pss, &ram_state->pss[RAM_CHANNEL_PRECOPY])) { + trace_postcopy_preempt_hit(pss->block->idstr, + pss->page << TARGET_PAGE_BITS); + return 0; + } + + do { + page_dirty =3D migration_bitmap_clear_dirty(rs, pss->block, pss->p= age); + + if (page_dirty) { + /* Be strict to return code; it must be 1, or what else? */ + if (ram_save_target_page(rs, pss) !=3D 1) { + error_report_once("%s: ram_save_target_page failed", __fun= c__); + ret =3D -1; + goto out; + } + sent =3D true; + } + pss_find_next_dirty(pss); + } while (pss_within_range(pss)); +out: + pss_host_page_finish(pss); + /* For urgent requests, flush immediately if sent */ + if (sent) { + qemu_fflush(pss->pss_channel); + } + return ret; +} + /** * ram_save_host_page: save a whole host page * --=20 2.32.0 From nobody Fri Apr 26 23:13:09 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1663719717; cv=none; d=zohomail.com; s=zohoarc; b=ewykG3X/OncNiyX+IYjDIPQ2yYSRj4fRcEWnWvZEi7nVI8WaOBTXczdAVE79qleJLaLup+3FppLon8MY+67i/t4bVmQpeD7r/NBTJ6xBtYRnl+UUS+Mo8IbceKOpqU8qP6OQHIyA8jyc0SWOUbMQNXvSumxCCHcMBI620byuang= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1663719717; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=eFM6UBQq9Gr7xD4s6za7TeZ1aBDahyr8gEmlZoCT4PI=; b=OGxtD/inwAqf4qvpWR/byggvcIT4vgPg3lVTsNSLBTUEszkqcg50xuXz73bau+wDk28ajkp0EobXZ+CDCiQRE31Tq4D8Lt1JqBkkMackTGadS+RaN2eUDk7xKgP22cKxDQULqYFNItVYP43S4Fgrbl27zXawIKkhQnecCF0ZpTo= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1663719717232821.0688329755284; Tue, 20 Sep 2022 17:21:57 -0700 (PDT) Received: from localhost ([::1]:33084 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1oanUx-0006Cz-P8 for importer@patchew.org; Tue, 20 Sep 2022 20:21:55 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:33456) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1oam6V-0005zo-3Z for qemu-devel@nongnu.org; Tue, 20 Sep 2022 18:52:35 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]:35243) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1oam6S-0001VP-Ir for qemu-devel@nongnu.org; Tue, 20 Sep 2022 18:52:34 -0400 Received: from mail-qv1-f69.google.com (mail-qv1-f69.google.com [209.85.219.69]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_128_GCM_SHA256) id us-mta-85-mDsLSB9aM-ugoO9R3YQddg-1; Tue, 20 Sep 2022 18:52:31 -0400 Received: by mail-qv1-f69.google.com with SMTP id n15-20020ad444af000000b004a2a341ad71so3049218qvt.15 for ; Tue, 20 Sep 2022 15:52:30 -0700 (PDT) Received: from localhost.localdomain (bras-base-aurron9127w-grc-46-70-31-27-79.dsl.bell.ca. [70.31.27.79]) by smtp.gmail.com with ESMTPSA id z12-20020ac87f8c000000b0035ba7012724sm596479qtj.70.2022.09.20.15.52.28 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Tue, 20 Sep 2022 15:52:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1663714352; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=eFM6UBQq9Gr7xD4s6za7TeZ1aBDahyr8gEmlZoCT4PI=; b=icb2oiMSZVvAubtk6JLf0pxssARoFKlVGsTe/V5BqfFlPKM+Pa7jq1740pN0oTYMTk6YUo ZBt4XbVRugpoZOOqPjTxWU+Pi+e/qTl5wz3ac94RA5OQft5EEAhR+7n4R3jTz9ZXi2tvgx IuqS3+sbEU4r1CSc3K5N0SRx/+rIITU= X-MC-Unique: mDsLSB9aM-ugoO9R3YQddg-1 X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date; bh=eFM6UBQq9Gr7xD4s6za7TeZ1aBDahyr8gEmlZoCT4PI=; b=S+TJaucxs+Pa4q7n1DhgloPXF7dcDGM50IwYIStPJpFYdZDcxj5CEr7+oSyWqgzqel U7dKWcQvlrSKXTSexX39+mXrh/qn/KM3vEOir7Gu3pGpvfDUHbC5RUqk41QQmLm1ktV4 T6RIJRqX5+wE961dXQGvadrLhcw/dwF6QFH8XAJ8sG2S/mwSOgQfIjqS+PsJZInFlU2g qo8hBKM0RIn+xphBavqzOpKrKAep68hr4bR0b+Gw3Feb0ldHg3rbp+d4EtmEPsXtqeex dfkYPo0z3YYdfidVGRsaIwv1t17rLpNhLxCni+APS/AZIGiGuAqIbFocD03jSPSbuXkx IPCg== X-Gm-Message-State: ACrzQf06npSFUPrlDYed/xdhAHecU2YrnJGnfVu25VpVy9LZ2pGw32YU M1a5K4hZOoCHjUbfb+cK9/lIc55So9VdFAAgOR0LPTAuxaA2qOdCGEM4dL7yDgamvghp7YmvzV1 wd7o/EoPRutVRfnSp2PSkJlMgbWAAs9nJtgZAilAkQSsX97Sr0E5b0G5oeRa1STer X-Received: by 2002:a05:620a:2956:b0:6ce:60f5:d887 with SMTP id n22-20020a05620a295600b006ce60f5d887mr18126442qkp.303.1663714350079; Tue, 20 Sep 2022 15:52:30 -0700 (PDT) X-Google-Smtp-Source: AMsMyM49lITWSIq6Qf4eStlCqcoinipwVYURNvenIv6S2ySvesFAJ4U0gTAtHMOq9uUg1mM4c/vx6Q== X-Received: by 2002:a05:620a:2956:b0:6ce:60f5:d887 with SMTP id n22-20020a05620a295600b006ce60f5d887mr18126421qkp.303.1663714349600; Tue, 20 Sep 2022 15:52:29 -0700 (PDT) From: Peter Xu To: qemu-devel@nongnu.org Cc: peterx@redhat.com, Manish Mishra , Juan Quintela , ani@anisinha.ca, Leonardo Bras Soares Passos , "Dr . David Alan Gilbert" , "Daniel P . Berrange" Subject: [PATCH 13/14] migration: Remove old preempt code around state maintainance Date: Tue, 20 Sep 2022 18:52:27 -0400 Message-Id: <20220920225227.49158-1-peterx@redhat.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20220920225106.48451-1-peterx@redhat.com> References: <20220920225106.48451-1-peterx@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=peterx@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1663719719039100001 Content-Type: text/plain; charset="utf-8" With the new code to send pages in rp-return thread, there's little help to keep lots of the old code on maintaining the preempt state in migration thread, because the new way should always be faster.. Then if we'll always send pages in the rp-return thread anyway, we don't need those logic to maintain preempt state anymore because now we serialize things using the mutex directly instead of using those fields. It's very unfortunate to have those code for a short period, but that's still one intermediate step that we noticed the next bottleneck on the migration thread. Now what we can do best is to drop unnecessary code as long as the new code is stable to reduce the burden. It's actually a good thing because the new "sending page in rp-return thread" model is (IMHO) even cleaner and with better performance. Remove the old code that was responsible for maintaining preempt states, at the meantime also remove x-postcopy-preempt-break-huge parameter because with concurrent sender threads we don't really need to break-huge anymore. Signed-off-by: Peter Xu Reviewed-by: Dr. David Alan Gilbert --- migration/migration.c | 2 - migration/ram.c | 258 +----------------------------------------- 2 files changed, 3 insertions(+), 257 deletions(-) diff --git a/migration/migration.c b/migration/migration.c index fae8fd378b..698fd94591 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -4399,8 +4399,6 @@ static Property migration_properties[] =3D { DEFINE_PROP_SIZE("announce-step", MigrationState, parameters.announce_step, DEFAULT_MIGRATE_ANNOUNCE_STEP), - DEFINE_PROP_BOOL("x-postcopy-preempt-break-huge", MigrationState, - postcopy_preempt_break_huge, true), DEFINE_PROP_STRING("tls-creds", MigrationState, parameters.tls_creds), DEFINE_PROP_STRING("tls-hostname", MigrationState, parameters.tls_host= name), DEFINE_PROP_STRING("tls-authz", MigrationState, parameters.tls_authz), diff --git a/migration/ram.c b/migration/ram.c index fd301d793c..f42efe02fc 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -343,20 +343,6 @@ struct RAMSrcPageRequest { QSIMPLEQ_ENTRY(RAMSrcPageRequest) next_req; }; =20 -typedef struct { - /* - * Cached ramblock/offset values if preempted. They're only meaningfu= l if - * preempted=3D=3Dtrue below. - */ - RAMBlock *ram_block; - unsigned long ram_page; - /* - * Whether a postcopy preemption just happened. Will be reset after - * precopy recovered to background migration. - */ - bool preempted; -} PostcopyPreemptState; - /* State of RAM for migration */ struct RAMState { /* QEMUFile used for this migration */ @@ -419,14 +405,6 @@ struct RAMState { /* Queue of outstanding page requests from the destination */ QemuMutex src_page_req_mutex; QSIMPLEQ_HEAD(, RAMSrcPageRequest) src_page_requests; - - /* Postcopy preemption informations */ - PostcopyPreemptState postcopy_preempt_state; - /* - * Current channel we're using on src VM. Only valid if postcopy-pree= mpt - * is enabled. - */ - unsigned int postcopy_channel; }; typedef struct RAMState RAMState; =20 @@ -434,11 +412,6 @@ static RAMState *ram_state; =20 static NotifierWithReturnList precopy_notifier_list; =20 -static void postcopy_preempt_reset(RAMState *rs) -{ - memset(&rs->postcopy_preempt_state, 0, sizeof(PostcopyPreemptState)); -} - /* Whether postcopy has queued requests? */ static bool postcopy_has_request(RAMState *rs) { @@ -544,9 +517,6 @@ static int ram_save_host_page_urgent(PageSearchStatus *= pss); static bool do_compress_ram_page(QEMUFile *f, z_stream *stream, RAMBlock *= block, ram_addr_t offset, uint8_t *source_buf); =20 -static void postcopy_preempt_restore(RAMState *rs, PageSearchStatus *pss, - bool postcopy_requested); - /* NOTE: page is the PFN not real ram_addr_t. */ static void pss_init(PageSearchStatus *pss, RAMBlock *rb, ram_addr_t page) { @@ -2062,55 +2032,6 @@ void ram_write_tracking_stop(void) } #endif /* defined(__linux__) */ =20 -/* - * Check whether two addr/offset of the ramblock falls onto the same host = huge - * page. Returns true if so, false otherwise. - */ -static bool offset_on_same_huge_page(RAMBlock *rb, uint64_t addr1, - uint64_t addr2) -{ - size_t page_size =3D qemu_ram_pagesize(rb); - - addr1 =3D ROUND_DOWN(addr1, page_size); - addr2 =3D ROUND_DOWN(addr2, page_size); - - return addr1 =3D=3D addr2; -} - -/* - * Whether a previous preempted precopy huge page contains current request= ed - * page? Returns true if so, false otherwise. - * - * This should really happen very rarely, because it means when we were se= nding - * during background migration for postcopy we're sending exactly the page= that - * some vcpu got faulted on on dest node. When it happens, we probably do= n't - * need to do much but drop the request, because we know right after we re= store - * the precopy stream it'll be serviced. It'll slightly affect the order = of - * postcopy requests to be serviced (e.g. it'll be the same as we move cur= rent - * request to the end of the queue) but it shouldn't be a big deal. The m= ost - * imporant thing is we can _never_ try to send a partial-sent huge page o= n the - * POSTCOPY channel again, otherwise that huge page will got "split brain"= on - * two channels (PRECOPY, POSTCOPY). - */ -static bool postcopy_preempted_contains(RAMState *rs, RAMBlock *block, - ram_addr_t offset) -{ - PostcopyPreemptState *state =3D &rs->postcopy_preempt_state; - - /* No preemption at all? */ - if (!state->preempted) { - return false; - } - - /* Not even the same ramblock? */ - if (state->ram_block !=3D block) { - return false; - } - - return offset_on_same_huge_page(block, offset, - state->ram_page << TARGET_PAGE_BITS); -} - /** * get_queued_page: unqueue a page from the postcopy requests * @@ -2150,20 +2071,7 @@ static bool get_queued_page(RAMState *rs, PageSearch= Status *pss) =20 } while (block && !dirty); =20 - if (block) { - /* See comment above postcopy_preempted_contains() */ - if (postcopy_preempted_contains(rs, block, offset)) { - trace_postcopy_preempt_hit(block->idstr, offset); - /* - * If what we preempted previously was exactly what we're - * requesting right now, restore the preempted precopy - * immediately, boosting its priority as it's requested by - * postcopy. - */ - postcopy_preempt_restore(rs, pss, true); - return true; - } - } else { + if (!block) { /* * Poll write faults too if background snapshot is enabled; that's * when we have vcpus got blocked by the write protected pages. @@ -2433,129 +2341,6 @@ static int ram_save_target_page(RAMState *rs, PageS= earchStatus *pss) return ram_save_page(rs, pss); } =20 -static bool postcopy_needs_preempt(RAMState *rs, PageSearchStatus *pss) -{ - MigrationState *ms =3D migrate_get_current(); - - /* Not enabled eager preempt? Then never do that. */ - if (!migrate_postcopy_preempt()) { - return false; - } - - /* If the user explicitly disabled breaking of huge page, skip */ - if (!ms->postcopy_preempt_break_huge) { - return false; - } - - /* If the ramblock we're sending is a small page? Never bother. */ - if (qemu_ram_pagesize(pss->block) =3D=3D TARGET_PAGE_SIZE) { - return false; - } - - /* Not in postcopy at all? */ - if (!migration_in_postcopy()) { - return false; - } - - /* - * If we're already handling a postcopy request, don't preempt as this= page - * has got the same high priority. - */ - if (pss->postcopy_requested) { - return false; - } - - /* If there's postcopy requests, then check it up! */ - return postcopy_has_request(rs); -} - -/* Returns true if we preempted precopy, false otherwise */ -static void postcopy_do_preempt(RAMState *rs, PageSearchStatus *pss) -{ - PostcopyPreemptState *p_state =3D &rs->postcopy_preempt_state; - - trace_postcopy_preempt_triggered(pss->block->idstr, pss->page); - - /* - * Time to preempt precopy. Cache current PSS into preempt state, so t= hat - * after handling the postcopy pages we can recover to it. We need to= do - * so because the dest VM will have partial of the precopy huge page k= ept - * over in its tmp huge page caches; better move on with it when we ca= n. - */ - p_state->ram_block =3D pss->block; - p_state->ram_page =3D pss->page; - p_state->preempted =3D true; -} - -/* Whether we're preempted by a postcopy request during sending a huge pag= e */ -static bool postcopy_preempt_triggered(RAMState *rs) -{ - return rs->postcopy_preempt_state.preempted; -} - -static void postcopy_preempt_restore(RAMState *rs, PageSearchStatus *pss, - bool postcopy_requested) -{ - PostcopyPreemptState *state =3D &rs->postcopy_preempt_state; - - assert(state->preempted); - - pss->block =3D state->ram_block; - pss->page =3D state->ram_page; - - /* Whether this is a postcopy request? */ - pss->postcopy_requested =3D postcopy_requested; - /* - * When restoring a preempted page, the old data resides in PRECOPY - * slow channel, even if postcopy_requested is set. So always use - * PRECOPY channel here. - */ - pss->postcopy_target_channel =3D RAM_CHANNEL_PRECOPY; - - trace_postcopy_preempt_restored(pss->block->idstr, pss->page); - - /* Reset preempt state, most importantly, set preempted=3D=3Dfalse */ - postcopy_preempt_reset(rs); -} - -static void postcopy_preempt_choose_channel(RAMState *rs, PageSearchStatus= *pss) -{ - MigrationState *s =3D migrate_get_current(); - unsigned int channel =3D pss->postcopy_target_channel; - QEMUFile *next; - - if (channel !=3D rs->postcopy_channel) { - if (channel =3D=3D RAM_CHANNEL_PRECOPY) { - next =3D s->to_dst_file; - } else { - next =3D s->postcopy_qemufile_src; - } - /* Update and cache the current channel */ - rs->f =3D next; - rs->postcopy_channel =3D channel; - - /* - * If channel switched, reset last_sent_block since the old sent b= lock - * may not be on the same channel. - */ - pss->last_sent_block =3D NULL; - - trace_postcopy_preempt_switch_channel(channel); - } - - trace_postcopy_preempt_send_host_page(pss->block->idstr, pss->page); -} - -/* We need to make sure rs->f always points to the default channel elsewhe= re */ -static void postcopy_preempt_reset_channel(RAMState *rs) -{ - if (postcopy_preempt_active()) { - rs->postcopy_channel =3D RAM_CHANNEL_PRECOPY; - rs->f =3D migrate_get_current()->to_dst_file; - trace_postcopy_preempt_reset_channel(); - } -} - /* Should be called before sending a host page */ static void pss_host_page_prepare(PageSearchStatus *pss) { @@ -2677,11 +2462,6 @@ static int ram_save_host_page(RAMState *rs, PageSear= chStatus *pss) pss_host_page_prepare(pss); =20 do { - if (postcopy_needs_preempt(rs, pss)) { - postcopy_do_preempt(rs, pss); - break; - } - page_dirty =3D migration_bitmap_clear_dirty(rs, pss->block, pss->p= age); /* * Properly yield the lock only in postcopy preempt mode because @@ -2723,19 +2503,6 @@ static int ram_save_host_page(RAMState *rs, PageSear= chStatus *pss) =20 pss_host_page_finish(pss); =20 - /* - * When with postcopy preempt mode, flush the data as soon as possible= for - * postcopy requests, because we've already sent a whole huge page, so= the - * dst node should already have enough resource to atomically filling = in - * the current missing page. - * - * More importantly, when using separate postcopy channel, we must do - * explicit flush or it won't flush until the buffer is full. - */ - if (migrate_postcopy_preempt() && pss->postcopy_requested) { - qemu_fflush(pss->pss_channel); - } - res =3D ram_save_release_protection(rs, pss, start_page); return (res < 0 ? res : pages); } @@ -2783,24 +2550,11 @@ static int ram_find_and_save_block(RAMState *rs) found =3D get_queued_page(rs, pss); =20 if (!found) { - /* - * Recover previous precopy ramblock/offset if postcopy has - * preempted precopy. Otherwise find the next dirty bit. - */ - if (postcopy_preempt_triggered(rs)) { - postcopy_preempt_restore(rs, pss, false); - found =3D true; - } else { - /* priority queue empty, so just search for something dirt= y */ - found =3D find_dirty_block(rs, pss, &again); - } + /* priority queue empty, so just search for something dirty */ + found =3D find_dirty_block(rs, pss, &again); } =20 if (found) { - /* Update rs->f with correct channel */ - if (postcopy_preempt_active()) { - postcopy_preempt_choose_channel(rs, pss); - } /* Cache rs->f in pss_channel (TODO: remove rs->f) */ pss->pss_channel =3D rs->f; pages =3D ram_save_host_page(rs, pss); @@ -2932,8 +2686,6 @@ static void ram_state_reset(RAMState *rs) rs->last_page =3D 0; rs->last_version =3D ram_list.version; rs->xbzrle_enabled =3D false; - postcopy_preempt_reset(rs); - rs->postcopy_channel =3D RAM_CHANNEL_PRECOPY; } =20 #define MAX_WAIT 50 /* ms, half buffered_file limit */ @@ -3577,8 +3329,6 @@ static int ram_save_iterate(QEMUFile *f, void *opaque) } qemu_mutex_unlock(&rs->bitmap_mutex); =20 - postcopy_preempt_reset_channel(rs); - /* * Must occur before EOS (or any QEMUFile operation) * because of RDMA protocol. @@ -3656,8 +3406,6 @@ static int ram_save_complete(QEMUFile *f, void *opaqu= e) return ret; } =20 - postcopy_preempt_reset_channel(rs); - ret =3D multifd_send_sync_main(rs->f); if (ret < 0) { return ret; --=20 2.32.0 From nobody Fri Apr 26 23:13:09 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1663718467; cv=none; d=zohomail.com; s=zohoarc; b=UO6k2vRPWbGQOOrGdQrz/96qbjGEPHVG0uGhN3itRVYSTPx/JioR7KKSb44qvR8V+qHir7JnFZRiAV4d+ekJadnhma0ec4geOyH5Azq8pqCZubNng6Ead3FDf2ROxaI3Za+aFvErp26bPz+y1705skYyC+yFj2tjR8JqE3+Vbsw= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1663718467; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=iwOXFkTQrTibzA65OyWTTWJhrtaPwhXcDaZWJsbWqCY=; b=ZsW/UW2Gslf0r2gx/xkBYt1WamwB8oCYtTCX+avB4I9Fz8SC9JJMARdgXl6fSLZWuH2uRJmaJvS3Lu+34X87bA0JMbCv5IM92jWXyRIbfoTd2IIwxOfw07+rvmI5eO1Uq/Y5kHuUAV/q908Dp+8cbHZ7jcbPacYDtgmYDVki6CY= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1663718467651261.3082459180631; Tue, 20 Sep 2022 17:01:07 -0700 (PDT) Received: from localhost ([::1]:58972 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1oanAo-0003DP-Hw for importer@patchew.org; Tue, 20 Sep 2022 20:01:06 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:41478) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1oam6W-00064z-Db for qemu-devel@nongnu.org; Tue, 20 Sep 2022 18:52:36 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:23949) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1oam6U-0001Vc-NX for qemu-devel@nongnu.org; Tue, 20 Sep 2022 18:52:36 -0400 Received: from mail-qv1-f72.google.com (mail-qv1-f72.google.com [209.85.219.72]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_128_GCM_SHA256) id us-mta-634-kut4LKlBN7SVF4f1XnnoGw-1; Tue, 20 Sep 2022 18:52:33 -0400 Received: by mail-qv1-f72.google.com with SMTP id y16-20020a0cec10000000b004a5df9e16c6so3046133qvo.1 for ; Tue, 20 Sep 2022 15:52:32 -0700 (PDT) Received: from localhost.localdomain (bras-base-aurron9127w-grc-46-70-31-27-79.dsl.bell.ca. [70.31.27.79]) by smtp.gmail.com with ESMTPSA id c3-20020a37e103000000b006ce3fcee2bdsm675903qkm.50.2022.09.20.15.52.30 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Tue, 20 Sep 2022 15:52:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1663714354; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=iwOXFkTQrTibzA65OyWTTWJhrtaPwhXcDaZWJsbWqCY=; b=L7EILORtyl+QD9Aa0rFatIOaCwqNDEdg1I9GTjZ0hDh5bNfVDAmN+GsMFW9w7nK/Hei8Ok TAAgACFcN5RAl1cnppy0oJloRKE8/RXOj0li3jVYMNGJdM3IvGjtFCwv7SpFb1B38Nf41B 6mutCyh8pmMPh7M1Gd30hx7Wl+NbZn4= X-MC-Unique: kut4LKlBN7SVF4f1XnnoGw-1 X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date; bh=iwOXFkTQrTibzA65OyWTTWJhrtaPwhXcDaZWJsbWqCY=; b=sziPieVvX7rRvwwjGvgeI4kFWAtNKj1cnL/YCX/O4hYeJZSDtOk8mvquuTu0isKW2n rKRDYpeJnmXWCGz97zkUraRXAvIWsAmor8gVEx9//dUeAMgo+fhOw6vxdmn9nO+vNryF yghC+8frZTfzj3MnX2nh3+jSXKAUMR76AJN2ppAwTvmcwSvBZyIgnw7KZI6WPvcBJg/B 6EgFL7SB26SA+Y0aKHPFAkvcSoQr6k/sC5QVRUjBnGKB08v49y4F37SdfijawSYE9xqV awNk1zW0PawMrLrE6chOT3YWvGdJ9ro1uQz8IEGfSouOfCADxXYUq6ntX8CgMsurqVCZ BpNg== X-Gm-Message-State: ACrzQf23ekGV0lGikO7/L3sIffIkf9yuoyAjp2+TkvNdxb5Dz6OwDsqC gaQMLu17rwHbo1kVEolw2pho/abwyZcTIY7VZShsc+7wmCAOoYFF2dRQzw7W9urbVvfFzQ/yI8G hXtZfcYq9S0b/7xQmkY6UP39uVHK3Dn9mR/JEcUbHnNk6J7VjoLV3nGvd3uN1sg5H X-Received: by 2002:a05:622a:590:b0:35c:f3f0:f8dc with SMTP id c16-20020a05622a059000b0035cf3f0f8dcmr7118635qtb.440.1663714352078; Tue, 20 Sep 2022 15:52:32 -0700 (PDT) X-Google-Smtp-Source: AMsMyM57KSLNMXFPEnUQIXIzBCWRd07AM8fRBSx+8D6xo4Ww7OM/aAImGeVQGpnth6Z7Azw8jaOAcA== X-Received: by 2002:a05:622a:590:b0:35c:f3f0:f8dc with SMTP id c16-20020a05622a059000b0035cf3f0f8dcmr7118618qtb.440.1663714351752; Tue, 20 Sep 2022 15:52:31 -0700 (PDT) From: Peter Xu To: qemu-devel@nongnu.org Cc: peterx@redhat.com, Manish Mishra , Juan Quintela , ani@anisinha.ca, Leonardo Bras Soares Passos , "Dr . David Alan Gilbert" , "Daniel P . Berrange" Subject: [PATCH 14/14] migration: Drop rs->f Date: Tue, 20 Sep 2022 18:52:29 -0400 Message-Id: <20220920225229.49212-1-peterx@redhat.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20220920225106.48451-1-peterx@redhat.com> References: <20220920225106.48451-1-peterx@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=peterx@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1663718469688100003 Content-Type: text/plain; charset="utf-8" Now with rs->pss we can already cache channels in pss->pss_channels. That pss_channel contains more infromation than rs->f because it's per-channel. So rs->f could be replaced by rss->pss[RAM_CHANNEL_PRECOPY].pss_channel, while rs->f itself is a bit vague now. Note that vanilla postcopy still send pages via pss[RAM_CHANNEL_PRECOPY], that's slightly confusing but it reflects the reality. Then, after the replacement we can safely drop rs->f. Signed-off-by: Peter Xu Reviewed-by: Dr. David Alan Gilbert --- migration/ram.c | 12 ++++-------- 1 file changed, 4 insertions(+), 8 deletions(-) diff --git a/migration/ram.c b/migration/ram.c index f42efe02fc..03bf2324ab 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -345,8 +345,6 @@ struct RAMSrcPageRequest { =20 /* State of RAM for migration */ struct RAMState { - /* QEMUFile used for this migration */ - QEMUFile *f; /* * PageSearchStatus structures for the channels when send pages. * Protected by the bitmap_mutex. @@ -2555,8 +2553,6 @@ static int ram_find_and_save_block(RAMState *rs) } =20 if (found) { - /* Cache rs->f in pss_channel (TODO: remove rs->f) */ - pss->pss_channel =3D rs->f; pages =3D ram_save_host_page(rs, pss); } } while (!pages && again); @@ -3112,7 +3108,7 @@ static void ram_state_resume_prepare(RAMState *rs, QE= MUFile *out) ram_state_reset(rs); =20 /* Update RAMState cache of output QEMUFile */ - rs->f =3D out; + rs->pss[RAM_CHANNEL_PRECOPY].pss_channel =3D out; =20 trace_ram_state_resume_prepare(pages); } @@ -3203,7 +3199,7 @@ static int ram_save_setup(QEMUFile *f, void *opaque) return -1; } } - (*rsp)->f =3D f; + (*rsp)->pss[RAM_CHANNEL_PRECOPY].pss_channel =3D f; =20 WITH_RCU_READ_LOCK_GUARD() { qemu_put_be64(f, ram_bytes_total_common(true) | RAM_SAVE_FLAG_MEM_= SIZE); @@ -3338,7 +3334,7 @@ static int ram_save_iterate(QEMUFile *f, void *opaque) out: if (ret >=3D 0 && migration_is_setup_or_active(migrate_get_current()->state)) { - ret =3D multifd_send_sync_main(rs->f); + ret =3D multifd_send_sync_main(rs->pss[RAM_CHANNEL_PRECOPY].pss_ch= annel); if (ret < 0) { return ret; } @@ -3406,7 +3402,7 @@ static int ram_save_complete(QEMUFile *f, void *opaqu= e) return ret; } =20 - ret =3D multifd_send_sync_main(rs->f); + ret =3D multifd_send_sync_main(rs->pss[RAM_CHANNEL_PRECOPY].pss_channe= l); if (ret < 0) { return ret; } --=20 2.32.0