From nobody Sun May 19 09:23:40 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1665525502; cv=none; d=zohomail.com; s=zohoarc; b=A5C6cKgOeOQg2Z/ZQFw9YDahSzDtr7o5iNu7nLW6UyCbBrpilAGXb5ecph6zOxo0yHP35r/R2rd1YO2hZXR02lt40dvwsvL2rKrgOhI4TOAXWp3IjvHbd8xS4N6i03fruVrSVX+UaGfjQ7Ake6O5yhqxup/azbqdIu3bSBuU1hM= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1665525502; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=f8P8rC6b0KNfCTgsUdutxidXaOSDJ5QtY1XIgKgEULM=; b=b5OMj/gO3iz12M2ysg5S1JBZmbpiPgQNvfcLX/tIHVFDHUoLidSekxyA+rnvDulmHXJJW+YMyU3WN9vil38zs6CXmSu8ISlP+dUaszq+J9eQ3DPpAwEwQNZ7fRH51wYbIRGY9R2oQ9x45Hfi4S9jpkwkxwo9kGFM8z246Y6rG4o= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1665525502572173.78192317857872; Tue, 11 Oct 2022 14:58:22 -0700 (PDT) Received: from localhost ([::1]:38998 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1oiNGX-0004ai-6f for importer@patchew.org; Tue, 11 Oct 2022 17:58:21 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:43742) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1oiNER-0001a7-Kg for qemu-devel@nongnu.org; Tue, 11 Oct 2022 17:56:11 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]:30345) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1oiNEL-0003UP-EZ for qemu-devel@nongnu.org; Tue, 11 Oct 2022 17:56:08 -0400 Received: from mail-qt1-f198.google.com (mail-qt1-f198.google.com [209.85.160.198]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_128_GCM_SHA256) id us-mta-491-yZTFJIuzNou2lc8Yhl0VbA-1; Tue, 11 Oct 2022 17:56:03 -0400 Received: by mail-qt1-f198.google.com with SMTP id fz10-20020a05622a5a8a00b0035ce18717daso9962542qtb.11 for ; Tue, 11 Oct 2022 14:56:03 -0700 (PDT) Received: from x1n.redhat.com (bras-base-aurron9127w-grc-46-70-31-27-79.dsl.bell.ca. [70.31.27.79]) by smtp.gmail.com with ESMTPSA id u10-20020a05620a430a00b006ce7cd81359sm13863230qko.110.2022.10.11.14.56.01 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 11 Oct 2022 14:56:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1665525364; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=f8P8rC6b0KNfCTgsUdutxidXaOSDJ5QtY1XIgKgEULM=; b=YZB8eUxnL3h3saxv9QepFgZ3MX4EPd8g5RV5UWstMRk9EQ/4mP1L0wFpTi/O0TSlp2rljr yxykt2dfuFPQUaZ2+Ed1iezBAQaN6mFCwmr1tcGBBefMhzZIs2mI7yr/T4I+HsfVN2fGu+ mCr03Ng+R+RiIK9eiTA86xUhXZpZHl0= X-MC-Unique: yZTFJIuzNou2lc8Yhl0VbA-1 X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=f8P8rC6b0KNfCTgsUdutxidXaOSDJ5QtY1XIgKgEULM=; b=AxHcRD52/pNqkv5wF1umEBQeXWi10rlSlrPwIwKlPTeNMdHeODOQTeVQRTwjovFeRW DVX7qQRST/wpYvMV6Q86XNTr5gsXYRcRVLgzujRPdo0oMIBamzaWR3zXkV2ZyOZ+4YL9 D/evhOdAVklQRiZ+RWwfI0fjAr3bHOA90eikO6Jhy98Hhl17NgXBvkLD0U3pPNa88EZW m6Q2A84o/xqDZqBGxf3F2x3a5mEjgQUYrKUx8p89cu9Rtng2ie3V1CpghqMf8CMkDtvk 7lXx1jUhGDFIyRl6qFYVm4mUVNQe2xAs18mKYRIShu/nu3DDYMzj/x1IkLLmXgsnUBFy kmqg== X-Gm-Message-State: ACrzQf1pqUuCTNiX3ZaDyBIR2XRLmlHAqLkAr91iYwgGLy8SwmdUgjIG 37OlEcQbFVwxqhkVQsX4aFpzUcbqnsDbzAULqxiB+pEZdmAcEkkkZ/KFr1qDEYBqRCdXK3fLO9G t+9cd4gVhlVyLSk1rHiZuF8yzWQjjPBCQ5uugjyUXi3dwENOx+JvHctXUufIQ2PvQ X-Received: by 2002:ac8:7d13:0:b0:35c:e54e:9c8 with SMTP id g19-20020ac87d13000000b0035ce54e09c8mr20944707qtb.320.1665525363015; Tue, 11 Oct 2022 14:56:03 -0700 (PDT) X-Google-Smtp-Source: AMsMyM5bneUi2UtgLyVELKcfQwLK1Ca6lcAPsgBvHOI1W/RY+2D2E1KTPs3fsU66ACLUatJYzx3l9Q== X-Received: by 2002:ac8:7d13:0:b0:35c:e54e:9c8 with SMTP id g19-20020ac87d13000000b0035ce54e09c8mr20944691qtb.320.1665525362766; Tue, 11 Oct 2022 14:56:02 -0700 (PDT) From: Peter Xu To: qemu-devel@nongnu.org Cc: "Dr . David Alan Gilbert" , peterx@redhat.com, "Daniel P . Berrange" , Juan Quintela , ani@anisinha.ca, Leonardo Bras Soares Passos , Manish Mishra Subject: [PATCH v2 01/15] migration: Take bitmap mutex when completing ram migration Date: Tue, 11 Oct 2022 17:55:45 -0400 Message-Id: <20221011215559.602584-2-peterx@redhat.com> X-Mailer: git-send-email 2.37.3 In-Reply-To: <20221011215559.602584-1-peterx@redhat.com> References: <20221011215559.602584-1-peterx@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=peterx@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1665525503348100001 Content-Type: text/plain; charset="utf-8" Any call to ram_find_and_save_block() needs to take the bitmap mutex. We used to not take it for most of ram_save_complete() because we thought we're the only one left using the bitmap, but it's not true after the preempt full patchset applied, since the return path can be taking it too. Signed-off-by: Peter Xu Reviewed-by: Dr. David Alan Gilbert Reviewed-by: Juan Quintela --- migration/ram.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/migration/ram.c b/migration/ram.c index 1338e47665..cfeb571800 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -3406,6 +3406,7 @@ static int ram_save_complete(QEMUFile *f, void *opaqu= e) /* try transferring iterative blocks of memory */ =20 /* flush all remaining blocks regardless of rate limiting */ + qemu_mutex_lock(&rs->bitmap_mutex); while (true) { int pages; =20 @@ -3419,6 +3420,7 @@ static int ram_save_complete(QEMUFile *f, void *opaqu= e) break; } } + qemu_mutex_unlock(&rs->bitmap_mutex); =20 flush_compressed_data(rs); ram_control_after_iterate(f, RAM_CONTROL_FINISH); --=20 2.37.3 From nobody Sun May 19 09:23:40 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1665525614; cv=none; d=zohomail.com; s=zohoarc; b=aOtO8PuCX7Bh8xF1w4PXNwk3Hwh6+xma51ehAaFaprIOB+IDDxh1lxBdM2KKL6KrYeYZAa3zuOWccgCPVcl7QYkB1CBavnQV0jozNAU8ndkGHSgjKek7eLbqy56/fzZKfFdVcyn84PmxPYwRh/3Di6fvGCHSZKR3B42tlLifxj4= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1665525614; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=1BBvt7GyaeaSk78u8ZaaoxGYE7eohdBbynVfXfB+oxE=; b=I9Hu2s/r5Lp8vyGtuCzJDs+dMZb/tTfl2OOyesblrCPSkkiBRvMhFfwcvALYnlfbtPfgM45KpJRAmHi8rxX6lIwuGkUukgrt62/WiXpROt8e0/MZDqxxsz/qbED1JUk9SC6eoTVcOPkRcVmCuPQj3fAnjm5PHiKGPqG8QyeDlrw= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1665525614385263.7638437350789; Tue, 11 Oct 2022 15:00:14 -0700 (PDT) Received: from localhost ([::1]:40262 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1oiNIK-0008Et-3n for importer@patchew.org; Tue, 11 Oct 2022 18:00:12 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:43746) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1oiNER-0001ae-NU for qemu-devel@nongnu.org; Tue, 11 Oct 2022 17:56:11 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:37793) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1oiNEN-0003UV-Gz for qemu-devel@nongnu.org; Tue, 11 Oct 2022 17:56:10 -0400 Received: from mail-qt1-f200.google.com (mail-qt1-f200.google.com [209.85.160.200]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_128_GCM_SHA256) id us-mta-408-ZJuormZ9N3uS9OaTIVArNg-1; Tue, 11 Oct 2022 17:56:05 -0400 Received: by mail-qt1-f200.google.com with SMTP id e24-20020ac84918000000b0039878b3c676so5677257qtq.6 for ; Tue, 11 Oct 2022 14:56:05 -0700 (PDT) Received: from x1n.redhat.com (bras-base-aurron9127w-grc-46-70-31-27-79.dsl.bell.ca. [70.31.27.79]) by smtp.gmail.com with ESMTPSA id u10-20020a05620a430a00b006ce7cd81359sm13863230qko.110.2022.10.11.14.56.02 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 11 Oct 2022 14:56:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1665525366; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=1BBvt7GyaeaSk78u8ZaaoxGYE7eohdBbynVfXfB+oxE=; b=Bb9ojVsz4fRudGRpJ6BvCqiY9zbPaadKah32Q0kbntFH2cUdL42Ci1m3XxM9eQX3R8G55m L9cX2OD1F3/GMg/wcdG/0dE7Dhep7VI1CM+RkThs3VmP3Ehu96PEYt8m/HRpUcgBmluIlz /Ig4bhn+0funNSbo0PHaakS1lBPA6bA= X-MC-Unique: ZJuormZ9N3uS9OaTIVArNg-1 X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=1BBvt7GyaeaSk78u8ZaaoxGYE7eohdBbynVfXfB+oxE=; b=M+T7YmUTLnye+5xCvQo+/0BnieubhjK8B31GChdKUKwCh8960TWZRNMD1dyRi10gjW PUOMuEzognNVsHVWXZG5PasENvxIIB8aBkXeoQN3iic6bge6YIHz5rbdAlbyVx1sx/8l s3qYzaEGBJF43jXaShDUTTKSwscvlQX2KhU/JvkkR3mG4lGZ6ey8BRmDM8hOMgMVw8O/ Gyn8k1joPRE/O25w3qSpBAp0HWp0HjT5yT7FIPC9zRQtyCqOfaV0JzAABtvX9G1i41NW ZYjW082yn2D4xN6hVbJ6XwJTlLgh3sN1FZ9s7zff0juf7fKsClhSDX3jZ1JpsUOcGTYt VuEg== X-Gm-Message-State: ACrzQf0kxt/TWvyvQ6s0EeXziOmtl3u6csJ0dzuhm/V9yzmbVt1lNH2+ p2gkAYdvKkmrp3ens9mHKHfHSfTvNgh0FtKklM0xe1wiAwJk3h37HtaydvIgdqqg0+ntACkg1m4 ZDiPIYXtw3jeOpgr50Jn7GlSbdlQCqS4/mCH0h4AhuUrArKtSwLulud98bFcRG+D9 X-Received: by 2002:a05:622a:5ce:b0:39a:17a2:ecb0 with SMTP id d14-20020a05622a05ce00b0039a17a2ecb0mr10611986qtb.160.1665525364434; Tue, 11 Oct 2022 14:56:04 -0700 (PDT) X-Google-Smtp-Source: AMsMyM4mZacYGfa6wgfiXxpHdrFqxHBfVnmGxWR7XzhYiEapmgHOI+eApzbcvbrJxa26wEWNOraorg== X-Received: by 2002:a05:622a:5ce:b0:39a:17a2:ecb0 with SMTP id d14-20020a05622a05ce00b0039a17a2ecb0mr10611965qtb.160.1665525364152; Tue, 11 Oct 2022 14:56:04 -0700 (PDT) From: Peter Xu To: qemu-devel@nongnu.org Cc: "Dr . David Alan Gilbert" , peterx@redhat.com, "Daniel P . Berrange" , Juan Quintela , ani@anisinha.ca, Leonardo Bras Soares Passos , Manish Mishra Subject: [PATCH v2 02/15] migration: Add postcopy_preempt_active() Date: Tue, 11 Oct 2022 17:55:46 -0400 Message-Id: <20221011215559.602584-3-peterx@redhat.com> X-Mailer: git-send-email 2.37.3 In-Reply-To: <20221011215559.602584-1-peterx@redhat.com> References: <20221011215559.602584-1-peterx@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=peterx@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1665525615895100001 Content-Type: text/plain; charset="utf-8" Add the helper to show that postcopy preempt enabled, meanwhile active. Reviewed-by: Dr. David Alan Gilbert Signed-off-by: Peter Xu Reviewed-by: Juan Quintela --- migration/ram.c | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-) diff --git a/migration/ram.c b/migration/ram.c index cfeb571800..7aaa843a21 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -162,6 +162,11 @@ out: return ret; } =20 +static bool postcopy_preempt_active(void) +{ + return migrate_postcopy_preempt() && migration_in_postcopy(); +} + bool ramblock_is_ignored(RAMBlock *block) { return !qemu_ram_is_migratable(block) || @@ -2433,7 +2438,7 @@ static void postcopy_preempt_choose_channel(RAMState = *rs, PageSearchStatus *pss) /* We need to make sure rs->f always points to the default channel elsewhe= re */ static void postcopy_preempt_reset_channel(RAMState *rs) { - if (migrate_postcopy_preempt() && migration_in_postcopy()) { + if (postcopy_preempt_active()) { rs->postcopy_channel =3D RAM_CHANNEL_PRECOPY; rs->f =3D migrate_get_current()->to_dst_file; trace_postcopy_preempt_reset_channel(); @@ -2471,7 +2476,7 @@ static int ram_save_host_page(RAMState *rs, PageSearc= hStatus *pss) return 0; } =20 - if (migrate_postcopy_preempt() && migration_in_postcopy()) { + if (postcopy_preempt_active()) { postcopy_preempt_choose_channel(rs, pss); } =20 --=20 2.37.3 From nobody Sun May 19 09:23:40 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1665525725; cv=none; d=zohomail.com; s=zohoarc; b=eQC4vHw9CBtKkoeadaNo/G0KcC8Da6rwXEqWuBw2Y+W53u6bIskcsC1rHmHtvceNMvi+5y1LEZfhBq6sRrr/IASL8reUj073L6KcqZJQegB7EzL8B5i3/VvYMbALsYpDwqxz8CugbxJf1lD5PcLShKQpyb9spdPox80LFqfMRhg= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1665525725; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=TbBeFWrldpqnKRU67CogO6lNOkEQaMTBiKveeGJjChs=; b=lS2mjJqH0JSnL2PpejdQ3SA8GS71t+5eC1H3Q8oK4w6rGLejd2HekF28UJl4SJHiFETu9LSt5/J/jCmuadrriZ8uaTvBPIdIYDWQtQJgFefJPMJexAxReGBQQGbd/aesNBRoVvgWbWSw7dGe6GVPBBvuhQFy2lnDTdopAoQFjQU= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1665525725327916.7224876726151; Tue, 11 Oct 2022 15:02:05 -0700 (PDT) Received: from localhost ([::1]:52534 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1oiNK8-0001oR-0d for importer@patchew.org; Tue, 11 Oct 2022 18:02:04 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:43748) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1oiNER-0001ag-OL for qemu-devel@nongnu.org; Tue, 11 Oct 2022 17:56:11 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:55362) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1oiNEO-0003Uf-2P for qemu-devel@nongnu.org; Tue, 11 Oct 2022 17:56:10 -0400 Received: from mail-qt1-f198.google.com (mail-qt1-f198.google.com [209.85.160.198]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_128_GCM_SHA256) id us-mta-16-MKvgiBWmP8mQEuhPXQ5JQg-1; Tue, 11 Oct 2022 17:56:06 -0400 Received: by mail-qt1-f198.google.com with SMTP id d1-20020ac80601000000b00388b0fc84beso9986488qth.3 for ; Tue, 11 Oct 2022 14:56:06 -0700 (PDT) Received: from x1n.redhat.com (bras-base-aurron9127w-grc-46-70-31-27-79.dsl.bell.ca. [70.31.27.79]) by smtp.gmail.com with ESMTPSA id u10-20020a05620a430a00b006ce7cd81359sm13863230qko.110.2022.10.11.14.56.04 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 11 Oct 2022 14:56:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1665525367; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=TbBeFWrldpqnKRU67CogO6lNOkEQaMTBiKveeGJjChs=; b=Ok+2VwkzFcVBA/UAnMTTATSscFss9S/wYEzE1gBseuUy/86z6bz/2M1kb5t6VaAaHtVecn iGRUzkDFOCJcqMQ0nzmwWrk3F+9piiG6LzdB18v4gmyQHQ4IEOW791jYsypjEyah2KEO20 pl33SyHRxf6nVcr7dawhJzgxuE53e5Y= X-MC-Unique: MKvgiBWmP8mQEuhPXQ5JQg-1 X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=TbBeFWrldpqnKRU67CogO6lNOkEQaMTBiKveeGJjChs=; b=WjA+VvTpU5b8i0cDowtnmZ7nRF+bN5BrEJZGstXx78u7UBpJOL0eXXutvtNUZEyB/o UWB4Y7N9PqkLL/almZBDTLo97fDEZge/fGGjr4vhCn/5G7+GkEmWX/Bcj7xiIMmpe83p jWYU2MIArQdb/a6UOKsUh08Dv2CNd32yhDEOna/mT/EadD+RjkXxd/4jA6oNd4uPgGoR lN9/605dfVic8x49mjzrzk22rN8ytNlyJoW2+V5ncmFZFW4e51xAjdoNGH5CJZzyDOaY Bc8gJB69w1/Y2N2E6991p7G2zi5Q34eCxl5tCTskgum5fYiwqjwOvxLwaDqLyMTr43Xu kG0A== X-Gm-Message-State: ACrzQf3L99Dh2En5DINKMrlEJ1KWCrFgp0SsoJ9eN3qRIKohGpf4yX/I P5WoV7YEFT6h/u1yw6a52qiKuqzuS1aMG4a1zo8TvhsSTT/wfQkceNRhvbrkJW1eiBAGaeC46s/ DlooKCfGdtrG5gwHK5BgEz4MKYxdvFOpk+l2RbF7kNgmtFQ5+3o1Izn0cISatWGK9 X-Received: by 2002:a37:b404:0:b0:6eb:ca54:db74 with SMTP id d4-20020a37b404000000b006ebca54db74mr12386595qkf.610.1665525365774; Tue, 11 Oct 2022 14:56:05 -0700 (PDT) X-Google-Smtp-Source: AMsMyM6Kr82CIPKYnGxMDXRogwxKcDcVT7L0K29zDWdACputhr5bXMQKH3gAEmLQ+3FVmwLcutHA5w== X-Received: by 2002:a37:b404:0:b0:6eb:ca54:db74 with SMTP id d4-20020a37b404000000b006ebca54db74mr12386581qkf.610.1665525365479; Tue, 11 Oct 2022 14:56:05 -0700 (PDT) From: Peter Xu To: qemu-devel@nongnu.org Cc: "Dr . David Alan Gilbert" , peterx@redhat.com, "Daniel P . Berrange" , Juan Quintela , ani@anisinha.ca, Leonardo Bras Soares Passos , Manish Mishra Subject: [PATCH v2 03/15] migration: Cleanup xbzrle zero page cache update logic Date: Tue, 11 Oct 2022 17:55:47 -0400 Message-Id: <20221011215559.602584-4-peterx@redhat.com> X-Mailer: git-send-email 2.37.3 In-Reply-To: <20221011215559.602584-1-peterx@redhat.com> References: <20221011215559.602584-1-peterx@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=peterx@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1665525726508100001 Content-Type: text/plain; charset="utf-8" The major change is to replace "!save_page_use_compression()" with "xbzrle_enabled" to make it clear. Reasonings: (1) When compression enabled, "!save_page_use_compression()" is exactly the same as checking "xbzrle_enabled". (2) When compression disabled, "!save_page_use_compression()" always return true. We used to try calling the xbzrle code, but after this change we won't, and we shouldn't need to. Since at it, drop the xbzrle_enabled check in xbzrle_cache_zero_page() because with this change it's not needed anymore. Reviewed-by: Dr. David Alan Gilbert Signed-off-by: Peter Xu Reviewed-by: Juan Quintela --- migration/ram.c | 6 +----- 1 file changed, 1 insertion(+), 5 deletions(-) diff --git a/migration/ram.c b/migration/ram.c index 7aaa843a21..562646609e 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -741,10 +741,6 @@ void mig_throttle_counter_reset(void) */ static void xbzrle_cache_zero_page(RAMState *rs, ram_addr_t current_addr) { - if (!rs->xbzrle_enabled) { - return; - } - /* We don't care if this fails to allocate a new cache page * as long as it updated an old one */ cache_insert(XBZRLE.cache, current_addr, XBZRLE.zero_target_page, @@ -2301,7 +2297,7 @@ static int ram_save_target_page(RAMState *rs, PageSea= rchStatus *pss) /* Must let xbzrle know, otherwise a previous (now 0'd) cached * page would be stale */ - if (!save_page_use_compression(rs)) { + if (rs->xbzrle_enabled) { XBZRLE_cache_lock(); xbzrle_cache_zero_page(rs, block->offset + offset); XBZRLE_cache_unlock(); --=20 2.37.3 From nobody Sun May 19 09:23:40 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1665526220; cv=none; d=zohomail.com; s=zohoarc; b=alNqlA82vAJM41RZWKaQn9QuPYN5bhKV4TPze72IOh5QGN9CtTNNIhDOABNwUGWcjlJQf5szxrYtJ10bnzhi1vwkRwwSVpKBGq/LJN8nnDYJ0pF2qU+yoIVOBRfue51nK7xuGHqJ7IqkfMxzcczW8PofYwPJ4oUAo4ITtdI9nXw= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1665526220; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=votgVLiMq+XbDSvDD8GekiBp/FHoNh2XK9AL1Ft2XZU=; b=RJ5CI1UcDLn0LaYn1ie7OFkGZiRJl/jNVADlKjROq6unvjwHOS4lE0VVr0mHDS3me1HS9xrxKwl4HbwiwJxl+01fnqKnMEml1YiiFQkrRyo3owHmraQhgS4Cy+Hgndxwqg2Bo9Fsauqz4YLeMoR/lyLz2jaWU8+vFzVxCmbbgOI= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1665526220537146.75002219313149; Tue, 11 Oct 2022 15:10:20 -0700 (PDT) Received: from localhost ([::1]:41442 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1oiNS5-0007XQ-12 for importer@patchew.org; Tue, 11 Oct 2022 18:10:17 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:43750) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1oiNER-0001ai-OT for qemu-devel@nongnu.org; Tue, 11 Oct 2022 17:56:11 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]:26653) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1oiNEP-0003V4-LG for qemu-devel@nongnu.org; Tue, 11 Oct 2022 17:56:10 -0400 Received: from mail-qv1-f71.google.com (mail-qv1-f71.google.com [209.85.219.71]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_128_GCM_SHA256) id us-mta-317-er5Vbj0_MGeK0fVpNhZD2A-1; Tue, 11 Oct 2022 17:56:07 -0400 Received: by mail-qv1-f71.google.com with SMTP id eu10-20020ad44f4a000000b004b18126c4bfso8690488qvb.11 for ; Tue, 11 Oct 2022 14:56:07 -0700 (PDT) Received: from x1n.redhat.com (bras-base-aurron9127w-grc-46-70-31-27-79.dsl.bell.ca. [70.31.27.79]) by smtp.gmail.com with ESMTPSA id u10-20020a05620a430a00b006ce7cd81359sm13863230qko.110.2022.10.11.14.56.05 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 11 Oct 2022 14:56:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1665525368; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=votgVLiMq+XbDSvDD8GekiBp/FHoNh2XK9AL1Ft2XZU=; b=TQsu3BmuntGHSSiQ0fAUXDRAh7/qlPwogO/7Z2wJ1sj5koj3+4moSxmy6m7GPe245mw/3f KD6/EnzyIhIHpb3JHKPIrtYoe8Avj7Hp+nONXqyIGscgTuaEy6rZ8BLBaN2yWYvSszNqEJ JmSnecoE+IMQgc7naQzztxqEpqHcJGI= X-MC-Unique: er5Vbj0_MGeK0fVpNhZD2A-1 X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=votgVLiMq+XbDSvDD8GekiBp/FHoNh2XK9AL1Ft2XZU=; b=Xv6jxcfHvlVx9zSVE12OmohmTxogYoPWaBh7oct8afb0V3jN31StcsrikNitjRjBY5 o1W8zr8ZfYzgO2n2+KdQ/q++frafCgA38laIL3Zdqr9q64KNSfPuGLyjjh7vyzWOAN/m ZY5u0zq6ai1jgLyJQkPRJBGd03UmClBjNZJ8xSVLxv2GomuyAI1y6azqrhvS5uXIi/q6 JiCOizSS5rrkWkz5dw4uGmWwVHDFGeOXCPl+PRJ5Lhxg2VbAiz5CdOtaX1EBYBoQJQj8 gtm3NRsHFnSmsbV0FIdBlzcGUP1kv+apwM/YupdmLguhcyUb9PkRdVVgHXE7Ef1AVC0U BgwQ== X-Gm-Message-State: ACrzQf1z0Y48QRFH30iBessT4WGkMs2xp9DVFdc3/9daaTi76p3MJmrS Vo3P2934fDtU+rzxvqDg46ATTGahz2S8K2qQKNtGHv8HjNcqfIlkZ/BKeewfOIsp2APwRJrSyN9 AoT6ZO9LYvUl26y5Kgkto9n//LtKcFY0Rtr/8lY2ahhBxUh26DQplMSUzMol0/+kg X-Received: by 2002:a37:9142:0:b0:6ea:e7fc:be79 with SMTP id t63-20020a379142000000b006eae7fcbe79mr13400797qkd.679.1665525367095; Tue, 11 Oct 2022 14:56:07 -0700 (PDT) X-Google-Smtp-Source: AMsMyM7h11qUtqQNEzWWjnmYqQ1xzLJT0ODqpbZb78xYaAeOuk6YMCDqMxfHr+C1zJRC34wqXVEyEw== X-Received: by 2002:a37:9142:0:b0:6ea:e7fc:be79 with SMTP id t63-20020a379142000000b006eae7fcbe79mr13400779qkd.679.1665525366821; Tue, 11 Oct 2022 14:56:06 -0700 (PDT) From: Peter Xu To: qemu-devel@nongnu.org Cc: "Dr . David Alan Gilbert" , peterx@redhat.com, "Daniel P . Berrange" , Juan Quintela , ani@anisinha.ca, Leonardo Bras Soares Passos , Manish Mishra Subject: [PATCH v2 04/15] migration: Trivial cleanup save_page_header() on same block check Date: Tue, 11 Oct 2022 17:55:48 -0400 Message-Id: <20221011215559.602584-5-peterx@redhat.com> X-Mailer: git-send-email 2.37.3 In-Reply-To: <20221011215559.602584-1-peterx@redhat.com> References: <20221011215559.602584-1-peterx@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=peterx@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1665526221890100001 Content-Type: text/plain; charset="utf-8" The 2nd check on RAM_SAVE_FLAG_CONTINUE is a bit redundant. Use a boolean to be clearer. Reviewed-by: Dr. David Alan Gilbert Signed-off-by: Peter Xu Reviewed-by: Juan Quintela --- migration/ram.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/migration/ram.c b/migration/ram.c index 562646609e..c90feedb13 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -661,14 +661,15 @@ static size_t save_page_header(RAMState *rs, QEMUFile= *f, RAMBlock *block, ram_addr_t offset) { size_t size, len; + bool same_block =3D (block =3D=3D rs->last_sent_block); =20 - if (block =3D=3D rs->last_sent_block) { + if (same_block) { offset |=3D RAM_SAVE_FLAG_CONTINUE; } qemu_put_be64(f, offset); size =3D 8; =20 - if (!(offset & RAM_SAVE_FLAG_CONTINUE)) { + if (!same_block) { len =3D strlen(block->idstr); qemu_put_byte(f, len); qemu_put_buffer(f, (uint8_t *)block->idstr, len); --=20 2.37.3 From nobody Sun May 19 09:23:40 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1665526455; cv=none; d=zohomail.com; s=zohoarc; b=BkkcVsEL8LKV57ndy7TrAWg1afyDx4h8Rsa3FaH4m5E4jdq73Rr15BjLUPhOnGssMZRV6L13T8+pp+eEVC0OldqsIJcx6DIyDUUHum7d6rlpC3Y+dcLnSLQz9JLvjTovqSFTm4JLLAbQvORCdWhXqDOHXGmvbF1d/+Qpnuhq6Jg= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1665526455; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=kn4M0YNPryH5ciDulBNpDt2G7aDC8JLQJ+Wg8SJYoGc=; b=cKg/3U3iszhKEzpL47G//7CbvN76k4roIvdZ/nWP0RglIIg4wqAWkO79PJa0+W8ASO/q/Rov5wjDTOMy9ujE7XDmN430C52dg9HTrGSpGLgpNdXiukdrgkF80E+LzgOeok4VAWsc41WtDd0JFS31IiByeyGQGe1iKI1KhC4k1Wg= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1665526455875868.3250965295463; Tue, 11 Oct 2022 15:14:15 -0700 (PDT) Received: from localhost ([::1]:52874 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1oiNVu-0006eT-TF for importer@patchew.org; Tue, 11 Oct 2022 18:14:14 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:43752) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1oiNET-0001cZ-24 for qemu-devel@nongnu.org; Tue, 11 Oct 2022 17:56:13 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]:23199) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1oiNER-0003Va-Ew for qemu-devel@nongnu.org; Tue, 11 Oct 2022 17:56:12 -0400 Received: from mail-qt1-f200.google.com (mail-qt1-f200.google.com [209.85.160.200]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_128_GCM_SHA256) id us-mta-533-x7vM62cqPI6pvtzaeuQ3KA-1; Tue, 11 Oct 2022 17:56:09 -0400 Received: by mail-qt1-f200.google.com with SMTP id n11-20020ac8674b000000b0039c9e248175so1705030qtp.14 for ; Tue, 11 Oct 2022 14:56:09 -0700 (PDT) Received: from x1n.redhat.com (bras-base-aurron9127w-grc-46-70-31-27-79.dsl.bell.ca. [70.31.27.79]) by smtp.gmail.com with ESMTPSA id u10-20020a05620a430a00b006ce7cd81359sm13863230qko.110.2022.10.11.14.56.06 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 11 Oct 2022 14:56:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1665525370; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=kn4M0YNPryH5ciDulBNpDt2G7aDC8JLQJ+Wg8SJYoGc=; b=XuBnFoMi7icTL4uNMP3TqY0R798If517J8P7gyb/ogG0tzYoPXLybAFdyNI1OGe3fRO4tV FFnpox5lzdn/z8x647N2GAVClK6ZvKLZLMjE+qq6r/ogSJwkqn8GCuSge9yhjeF2Kvec3v nX4Mg7PXGr+/PFwNOm44RfcFf0pNzQ8= X-MC-Unique: x7vM62cqPI6pvtzaeuQ3KA-1 X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=kn4M0YNPryH5ciDulBNpDt2G7aDC8JLQJ+Wg8SJYoGc=; b=omq2UYn3cWIwmZ9GkVhXjJLwgoIqsvcx5XMPjUUhtWa5ErAeIZEBKxaO+iQc+rq07z Ye97umWYxfGdi+ORTOJElN/mbBXJz8ziXsyUnkNLp8lx6y5AGij3Jv2kD4SknTHcXwre fB4F1eNPEQiG9TGCrNJoAiC1ZO5CA0Q9J/WPWFihmM3C6PDrSDu+35q8jCsfqjd0Oh5f uB7r/XI4nrjZ18NzhCf2T25KuMnQtNntYlVHFNK+9YBQIwVQrVPWihauLOZK+1nXFimr QIoH8gXX4yU72SL6D+veVbwHPmrmDHdyH2w5VthHq8iSgguZgCBveN/b+7X8Mgrn8Q6X q0IA== X-Gm-Message-State: ACrzQf3tapZLV8FT6QJz37nEF/F2GlClrbFpgO3iiFZGKhg+TPudyb1m ZKgEb1xdPKTzbzMnxsfH4zQR9IKK5EGvQAEOONDnoA//Mz/uhl8RE2ATIGYHiX78zEwuTnsQKz1 xD5JIuuI1550Fqwczqm8CdGndoJ2lSLO2H7JzE0RNIGlcTWEp6/UVY+k3AnlJldAz X-Received: by 2002:a05:622a:83:b0:35d:44c1:e026 with SMTP id o3-20020a05622a008300b0035d44c1e026mr21234968qtw.298.1665525368386; Tue, 11 Oct 2022 14:56:08 -0700 (PDT) X-Google-Smtp-Source: AMsMyM7Mn41/fpr8S/DoeL67MKZDPQyrKJ1Pfmja/ENViUp8J81Tc7FFZVPnugJ7zYI81O8nFCkJsA== X-Received: by 2002:a05:622a:83:b0:35d:44c1:e026 with SMTP id o3-20020a05622a008300b0035d44c1e026mr21234946qtw.298.1665525368111; Tue, 11 Oct 2022 14:56:08 -0700 (PDT) From: Peter Xu To: qemu-devel@nongnu.org Cc: "Dr . David Alan Gilbert" , peterx@redhat.com, "Daniel P . Berrange" , Juan Quintela , ani@anisinha.ca, Leonardo Bras Soares Passos , Manish Mishra Subject: [PATCH v2 05/15] migration: Remove RAMState.f references in compression code Date: Tue, 11 Oct 2022 17:55:49 -0400 Message-Id: <20221011215559.602584-6-peterx@redhat.com> X-Mailer: git-send-email 2.37.3 In-Reply-To: <20221011215559.602584-1-peterx@redhat.com> References: <20221011215559.602584-1-peterx@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=peterx@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1665526457341100001 Content-Type: text/plain; charset="utf-8" Removing referencing to RAMState.f in compress_page_with_multi_thread() and flush_compressed_data(). Compression code by default isn't compatible with having >1 channels (or it won't currently know which channel to flush the compressed data), so to make it simple we always flush on the default to_dst_file port until someone wants to add >1 ports support, as rs->f right now can really change (after postcopy preempt is introduced). There should be no functional change at all after patch applied, since as long as rs->f referenced in compression code, it must be to_dst_file. Reviewed-by: Dr. David Alan Gilbert Signed-off-by: Peter Xu Reviewed-by: Juan Quintela --- migration/ram.c | 12 +++++++----- 1 file changed, 7 insertions(+), 5 deletions(-) diff --git a/migration/ram.c b/migration/ram.c index c90feedb13..b9ac2d6921 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -1461,6 +1461,7 @@ static bool save_page_use_compression(RAMState *rs); =20 static void flush_compressed_data(RAMState *rs) { + MigrationState *ms =3D migrate_get_current(); int idx, len, thread_count; =20 if (!save_page_use_compression(rs)) { @@ -1479,7 +1480,7 @@ static void flush_compressed_data(RAMState *rs) for (idx =3D 0; idx < thread_count; idx++) { qemu_mutex_lock(&comp_param[idx].mutex); if (!comp_param[idx].quit) { - len =3D qemu_put_qemu_file(rs->f, comp_param[idx].file); + len =3D qemu_put_qemu_file(ms->to_dst_file, comp_param[idx].fi= le); /* * it's safe to fetch zero_page without holding comp_done_lock * as there is no further request submitted to the thread, @@ -1498,11 +1499,11 @@ static inline void set_compress_params(CompressPara= m *param, RAMBlock *block, param->offset =3D offset; } =20 -static int compress_page_with_multi_thread(RAMState *rs, RAMBlock *block, - ram_addr_t offset) +static int compress_page_with_multi_thread(RAMBlock *block, ram_addr_t off= set) { int idx, thread_count, bytes_xmit =3D -1, pages =3D -1; bool wait =3D migrate_compress_wait_thread(); + MigrationState *ms =3D migrate_get_current(); =20 thread_count =3D migrate_compress_threads(); qemu_mutex_lock(&comp_done_lock); @@ -1510,7 +1511,8 @@ retry: for (idx =3D 0; idx < thread_count; idx++) { if (comp_param[idx].done) { comp_param[idx].done =3D false; - bytes_xmit =3D qemu_put_qemu_file(rs->f, comp_param[idx].file); + bytes_xmit =3D qemu_put_qemu_file(ms->to_dst_file, + comp_param[idx].file); qemu_mutex_lock(&comp_param[idx].mutex); set_compress_params(&comp_param[idx], block, offset); qemu_cond_signal(&comp_param[idx].cond); @@ -2263,7 +2265,7 @@ static bool save_compress_page(RAMState *rs, RAMBlock= *block, ram_addr_t offset) return false; } =20 - if (compress_page_with_multi_thread(rs, block, offset) > 0) { + if (compress_page_with_multi_thread(block, offset) > 0) { return true; } =20 --=20 2.37.3 From nobody Sun May 19 09:23:40 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1665525933; cv=none; d=zohomail.com; s=zohoarc; b=jV3FSkBOLUYccOXslfzHpIJ/osnIGQJQm42fHhYOhN8XAb/vcNz2caqwRFzsEuJK0ILJj/9w4GxOXxsW8h5hMVIQUY7XyKh/NNLEoIbhbM3oTuQ5HjrFE5Yuw5RjJkMQWyr+ozVDgzWTbWfOJYUHfZw/99LHmhihSVP1i/debHA= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1665525933; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=1W8eTe6R8APkrAjKrbrJHIMon0AQ1L7/5Yd/T0XmgZs=; b=cRTnKrJs67EKuhc7fjdvgEK7sulS16EgANG32Cztnpecx5Dc4nlkklLSKN+2XSWunkbLNYmklahC2tWHx1sIHr3rPuO5aCCd9XXj0LjMvPgoiP6t/RQy4mRZ3iWa5nunbwNuDpiI7SgGJj3PB4I3wR4U9VmlbC5IDqRTfEOhhWk= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1665525933167511.88252319816854; Tue, 11 Oct 2022 15:05:33 -0700 (PDT) Received: from localhost ([::1]:40556 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1oiNNT-0006NL-OL for importer@patchew.org; Tue, 11 Oct 2022 18:05:31 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:53864) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1oiNEV-0001gS-Pw for qemu-devel@nongnu.org; Tue, 11 Oct 2022 17:56:17 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:22978) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1oiNEU-0003WJ-6C for qemu-devel@nongnu.org; Tue, 11 Oct 2022 17:56:15 -0400 Received: from mail-qv1-f72.google.com (mail-qv1-f72.google.com [209.85.219.72]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_128_GCM_SHA256) id us-mta-332-FTF0VCZYMqSYSYNxXgXalg-1; Tue, 11 Oct 2022 17:56:10 -0400 Received: by mail-qv1-f72.google.com with SMTP id jn13-20020ad45ded000000b004b1d055fbc7so8685439qvb.2 for ; Tue, 11 Oct 2022 14:56:10 -0700 (PDT) Received: from x1n.redhat.com (bras-base-aurron9127w-grc-46-70-31-27-79.dsl.bell.ca. [70.31.27.79]) by smtp.gmail.com with ESMTPSA id u10-20020a05620a430a00b006ce7cd81359sm13863230qko.110.2022.10.11.14.56.08 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 11 Oct 2022 14:56:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1665525373; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=1W8eTe6R8APkrAjKrbrJHIMon0AQ1L7/5Yd/T0XmgZs=; b=L5SOT4gZbk/wyrn4Q1i0Nxgs41PNZj13oHUlM1eLHtyTV0y7fudFaYXa54yi+q88TDVt7w ZgAHLlc/RC608o6f4a1yWGHM9rQ8tHB8cv3GkdpfRizMPK5v9lBNvT9ujkc03+xrlhHWzz 86SRPYRwpQGf9RB96R0w7edVGmrvkeI= X-MC-Unique: FTF0VCZYMqSYSYNxXgXalg-1 X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=1W8eTe6R8APkrAjKrbrJHIMon0AQ1L7/5Yd/T0XmgZs=; b=5k+MpbWRDBEbcJkZ4fUjJG/PY6RnhAnQBs0S/o/tnBsHXp8f0ypjDCVCJvdVlo3tzq iOjQ8Ukoztkvkgf7hV/cLtJ6sioL1lHOHZY8GyTrjI/32OV+sTwKF5ZpfVrh9ChAukUD /SHdd5dPU750bn7wTWRhYIT2DLiZa8OK+nNE9BT5qrFYv+8EDvQVCXmtj5N4jMT7viQ0 fz27SsWJbiOjTPVNTzt+bmFEPBEBvrd9dcURgjAlbTNerJ1ogbMWHqELxfpcz16T2jSc khiBWh8ceP/REve0QezAKhh2Iiir15m+IFSlfFdMNPHWjZh+q/BTAZknLggBtkctTOjT JUkg== X-Gm-Message-State: ACrzQf3Cu79vqDdpEwk+DmDS3eM/wzMfBJKhmUbEEAsC4YhCu6c3s+rU p9pwj7N1pZOCcw2JPU5A19DxNu3k9cMT6sz2WAX28Uo8F7HdWZZGO5INhk/twC4F/lJ9aBW8YU4 jDd6YcF+qAWbVd41anuk/oud+HV5botpV57gAuKoPGqLAcx28ryB3pGOehTM2YfKt X-Received: by 2002:a05:620a:2056:b0:6ec:52f0:f2cb with SMTP id d22-20020a05620a205600b006ec52f0f2cbmr11051018qka.191.1665525369961; Tue, 11 Oct 2022 14:56:09 -0700 (PDT) X-Google-Smtp-Source: AMsMyM5VlFAuUgYD9nYhm+oGmleOZKg7M0Xgcg6YhGtsMaf6ZRUnKJnV6E4mlwfyq0PGPQEIcYWZXg== X-Received: by 2002:a05:620a:2056:b0:6ec:52f0:f2cb with SMTP id d22-20020a05620a205600b006ec52f0f2cbmr11051001qka.191.1665525369629; Tue, 11 Oct 2022 14:56:09 -0700 (PDT) From: Peter Xu To: qemu-devel@nongnu.org Cc: "Dr . David Alan Gilbert" , peterx@redhat.com, "Daniel P . Berrange" , Juan Quintela , ani@anisinha.ca, Leonardo Bras Soares Passos , Manish Mishra Subject: [PATCH v2 06/15] migration: Yield bitmap_mutex properly when sending/sleeping Date: Tue, 11 Oct 2022 17:55:50 -0400 Message-Id: <20221011215559.602584-7-peterx@redhat.com> X-Mailer: git-send-email 2.37.3 In-Reply-To: <20221011215559.602584-1-peterx@redhat.com> References: <20221011215559.602584-1-peterx@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=peterx@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1665525934234100001 Content-Type: text/plain; charset="utf-8" Don't take the bitmap mutex when sending pages, or when being throttled by migration_rate_limit() (which is a bit tricky to call it here in ram code, but seems still helpful). It prepares for the possibility of concurrently sending pages in >1 threads using the function ram_save_host_page() because all threads may need the bitmap_mutex to operate on bitmaps, so that either sendmsg() or any kind of qemu_sem_wait() blocking for one thread will not block the other from progressing. Signed-off-by: Peter Xu Reviewed-by: Dr. David Alan Gilbert Reviewed-by: Juan Quintela --- migration/ram.c | 41 ++++++++++++++++++++++++++++++----------- 1 file changed, 30 insertions(+), 11 deletions(-) diff --git a/migration/ram.c b/migration/ram.c index b9ac2d6921..578ad8d70a 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -2462,6 +2462,7 @@ static void postcopy_preempt_reset_channel(RAMState *= rs) */ static int ram_save_host_page(RAMState *rs, PageSearchStatus *pss) { + bool page_dirty, preempt_active =3D postcopy_preempt_active(); int tmppages, pages =3D 0; size_t pagesize_bits =3D qemu_ram_pagesize(pss->block) >> TARGET_PAGE_BITS; @@ -2485,22 +2486,40 @@ static int ram_save_host_page(RAMState *rs, PageSea= rchStatus *pss) break; } =20 - /* Check the pages is dirty and if it is send it */ - if (migration_bitmap_clear_dirty(rs, pss->block, pss->page)) { - tmppages =3D ram_save_target_page(rs, pss); - if (tmppages < 0) { - return tmppages; - } + page_dirty =3D migration_bitmap_clear_dirty(rs, pss->block, pss->p= age); =20 - pages +=3D tmppages; + /* Check the pages is dirty and if it is send it */ + if (page_dirty) { /* - * Allow rate limiting to happen in the middle of huge pages if - * something is sent in the current iteration. + * Properly yield the lock only in postcopy preempt mode + * because both migration thread and rp-return thread can + * operate on the bitmaps. */ - if (pagesize_bits > 1 && tmppages > 0) { - migration_rate_limit(); + if (preempt_active) { + qemu_mutex_unlock(&rs->bitmap_mutex); + } + tmppages =3D ram_save_target_page(rs, pss); + if (tmppages >=3D 0) { + pages +=3D tmppages; + /* + * Allow rate limiting to happen in the middle of huge pag= es if + * something is sent in the current iteration. + */ + if (pagesize_bits > 1 && tmppages > 0) { + migration_rate_limit(); + } } + if (preempt_active) { + qemu_mutex_lock(&rs->bitmap_mutex); + } + } else { + tmppages =3D 0; } + + if (tmppages < 0) { + return tmppages; + } + pss->page =3D migration_bitmap_find_dirty(rs, pss->block, pss->pag= e); } while ((pss->page < hostpage_boundary) && offset_in_ramblock(pss->block, --=20 2.37.3 From nobody Sun May 19 09:23:40 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1665525973; cv=none; d=zohomail.com; s=zohoarc; b=AVyIA66ogGH0MXyu+ruB9mZHvht+Z0fhzA0NngvSVugRs9RkiceVdCDbW23Y4YCePNr/z9mbFWwDVB+bHCd+9fDaXWTIrrp5WXjxeLcKDLp/HhEsfy2fz03YQrllxCqv/o0cZAgnkolMEjQBelKkx7cIltKWIRb2LiOGvMVER+I= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1665525973; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=CyCTybZa1i/wY9A4srjTud0msjBpnlVizEohyZvnA0g=; b=Pt8WNsv6PIF2UA5FptiMiYOP/6xhcNFJc82GNDic4ncL0GbHp8e1eI+JQFihAiOUeONtcOTyG+TqouKpZWnIOPh7nlU89o7G7gbFrhgnymz28bIEaI1jIKJlxY3v2I4vnmV6Gri6WU1yUnFEwTaOuIPZWfrKq2rZOzg62R2NJ7g= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1665525973638791.3660709321274; Tue, 11 Oct 2022 15:06:13 -0700 (PDT) Received: from localhost ([::1]:43188 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1oiNO8-0008Kl-IY for importer@patchew.org; Tue, 11 Oct 2022 18:06:12 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:53862) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1oiNEV-0001gA-Lx for qemu-devel@nongnu.org; Tue, 11 Oct 2022 17:56:17 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:49362) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1oiNET-0003WE-NV for qemu-devel@nongnu.org; Tue, 11 Oct 2022 17:56:15 -0400 Received: from mail-qk1-f197.google.com (mail-qk1-f197.google.com [209.85.222.197]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_128_GCM_SHA256) id us-mta-172-u4L65gEWMQq8QU51zQTPAw-1; Tue, 11 Oct 2022 17:56:12 -0400 Received: by mail-qk1-f197.google.com with SMTP id h9-20020a05620a244900b006ee944ec451so435907qkn.13 for ; Tue, 11 Oct 2022 14:56:11 -0700 (PDT) Received: from x1n.redhat.com (bras-base-aurron9127w-grc-46-70-31-27-79.dsl.bell.ca. [70.31.27.79]) by smtp.gmail.com with ESMTPSA id u10-20020a05620a430a00b006ce7cd81359sm13863230qko.110.2022.10.11.14.56.09 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 11 Oct 2022 14:56:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1665525373; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=CyCTybZa1i/wY9A4srjTud0msjBpnlVizEohyZvnA0g=; b=ShAmzA2gZ2HBEDrAzb0M8EggNT07eailTqtXmS1YYpGdZe7REqEoAk+HXaRO7UbxX2E0KR x5ZuS5MTSgIOBliC9rgUf3D7DTDk8ZtKdZ294Ip5iI4pdR9rPYCKJmDiUOBcGmAE91LDqM F8E4TKFqXhZyUjLH3TgjJb75SbRpQuM= X-MC-Unique: u4L65gEWMQq8QU51zQTPAw-1 X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=CyCTybZa1i/wY9A4srjTud0msjBpnlVizEohyZvnA0g=; b=4PoTHFUBildem+cQhS+k8+MxrAAbx4VwUmv1kRiJdWH8ha4+3TNRCSJ8j4xnW6ycyk WDQxlnLwkOa13277UaJogcpeB6UZDkYKoNEPgUoifhN5QOZrrcQBfnydrVLn9MGyFhPC Qz7slyrSys6quOA18LpZuP2iNAFyvCsIJ7USz11w6B1/0H8uW/F15N85xydjgT94w4J5 e08bQZ61Tm7DBsy2voM9ENaWCYdC7wMx19zuOye6Md7x0W1RHMFE32owop/C3IaFXbHJ FfXx3LcR+BdqZNmjUv0bBlEb+NbhgzdJysKTmwfiBhygXNqc9cZ28yKPgKYATr3gjKqg pe0g== X-Gm-Message-State: ACrzQf1RNhIpwfZQ4wulCjfFGNrU3cByqoD/xTek8ymOlgwMROnTFiE+ /Bdjp8+ie5yHgOr03I2Eq/QU1ExScNIO2m/LqRwr0IUMIqVI3c7I0ZnK4xCsd16am//LIQFZQtF 7fQJdB9xK4REmnVTLsXeVqMV2RjPr5t1xMTZ7bjYqGRqes1+hU8jOTAyqiDw2xakz X-Received: by 2002:ac8:5947:0:b0:399:67fa:652 with SMTP id 7-20020ac85947000000b0039967fa0652mr11657681qtz.519.1665525371237; Tue, 11 Oct 2022 14:56:11 -0700 (PDT) X-Google-Smtp-Source: AMsMyM4gXIaoQ9MPBcALjZFt9tJ6wGE2WvOm1Za+mZhcvTzrEoqwYZa5NFn+VIKS2TEckDR429z5PA== X-Received: by 2002:ac8:5947:0:b0:399:67fa:652 with SMTP id 7-20020ac85947000000b0039967fa0652mr11657660qtz.519.1665525370927; Tue, 11 Oct 2022 14:56:10 -0700 (PDT) From: Peter Xu To: qemu-devel@nongnu.org Cc: "Dr . David Alan Gilbert" , peterx@redhat.com, "Daniel P . Berrange" , Juan Quintela , ani@anisinha.ca, Leonardo Bras Soares Passos , Manish Mishra Subject: [PATCH v2 07/15] migration: Use atomic ops properly for page accountings Date: Tue, 11 Oct 2022 17:55:51 -0400 Message-Id: <20221011215559.602584-8-peterx@redhat.com> X-Mailer: git-send-email 2.37.3 In-Reply-To: <20221011215559.602584-1-peterx@redhat.com> References: <20221011215559.602584-1-peterx@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=peterx@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1665525974345100001 Content-Type: text/plain; charset="utf-8" To prepare for thread-safety on page accountings, at least below counters need to be accessed only atomically, they are: ram_counters.transferred ram_counters.duplicate ram_counters.normal ram_counters.postcopy_bytes There are a lot of other counters but they won't be accessed outside migration thread, then they're still safe to be accessed without atomic ops. Reviewed-by: Dr. David Alan Gilbert Signed-off-by: Peter Xu Reviewed-by: Juan Quintela --- migration/migration.c | 10 +++++----- migration/multifd.c | 4 ++-- migration/ram.c | 40 ++++++++++++++++++++++++---------------- migration/ram.h | 20 ++++++++++++++++++++ 4 files changed, 51 insertions(+), 23 deletions(-) diff --git a/migration/migration.c b/migration/migration.c index ef00bff0b3..4364813d82 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -1048,13 +1048,13 @@ static void populate_ram_info(MigrationInfo *info, = MigrationState *s) =20 info->has_ram =3D true; info->ram =3D g_malloc0(sizeof(*info->ram)); - info->ram->transferred =3D ram_counters.transferred; + info->ram->transferred =3D stat64_get(&ram_atomic_counters.transferred= ); info->ram->total =3D ram_bytes_total(); - info->ram->duplicate =3D ram_counters.duplicate; + info->ram->duplicate =3D stat64_get(&ram_atomic_counters.duplicate); /* legacy value. It is not used anymore */ info->ram->skipped =3D 0; - info->ram->normal =3D ram_counters.normal; - info->ram->normal_bytes =3D ram_counters.normal * page_size; + info->ram->normal =3D stat64_get(&ram_atomic_counters.normal); + info->ram->normal_bytes =3D info->ram->normal * page_size; info->ram->mbps =3D s->mbps; info->ram->dirty_sync_count =3D ram_counters.dirty_sync_count; info->ram->dirty_sync_missed_zero_copy =3D @@ -1065,7 +1065,7 @@ static void populate_ram_info(MigrationInfo *info, Mi= grationState *s) info->ram->pages_per_second =3D s->pages_per_second; info->ram->precopy_bytes =3D ram_counters.precopy_bytes; info->ram->downtime_bytes =3D ram_counters.downtime_bytes; - info->ram->postcopy_bytes =3D ram_counters.postcopy_bytes; + info->ram->postcopy_bytes =3D stat64_get(&ram_atomic_counters.postcopy= _bytes); =20 if (migrate_use_xbzrle()) { info->has_xbzrle_cache =3D true; diff --git a/migration/multifd.c b/migration/multifd.c index 586ddc9d65..6b1dc7c889 100644 --- a/migration/multifd.c +++ b/migration/multifd.c @@ -437,7 +437,7 @@ static int multifd_send_pages(QEMUFile *f) + p->packet_len; qemu_file_acct_rate_limit(f, transferred); ram_counters.multifd_bytes +=3D transferred; - ram_counters.transferred +=3D transferred; + stat64_add(&ram_atomic_counters.transferred, transferred); qemu_mutex_unlock(&p->mutex); qemu_sem_post(&p->sem); =20 @@ -612,7 +612,7 @@ int multifd_send_sync_main(QEMUFile *f) p->pending_job++; qemu_file_acct_rate_limit(f, p->packet_len); ram_counters.multifd_bytes +=3D p->packet_len; - ram_counters.transferred +=3D p->packet_len; + stat64_add(&ram_atomic_counters.transferred, p->packet_len); qemu_mutex_unlock(&p->mutex); qemu_sem_post(&p->sem); =20 diff --git a/migration/ram.c b/migration/ram.c index 578ad8d70a..f5a86265c7 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -425,18 +425,25 @@ uint64_t ram_bytes_remaining(void) 0; } =20 +/* + * NOTE: not all stats in ram_counters are used in reality. See comments + * for struct MigrationAtomicStats. The ultimate result of ram migration + * counters will be a merged version with both ram_counters and the atomic + * fields in ram_atomic_counters. + */ MigrationStats ram_counters; +MigrationAtomicStats ram_atomic_counters; =20 static void ram_transferred_add(uint64_t bytes) { if (runstate_is_running()) { ram_counters.precopy_bytes +=3D bytes; } else if (migration_in_postcopy()) { - ram_counters.postcopy_bytes +=3D bytes; + stat64_add(&ram_atomic_counters.postcopy_bytes, bytes); } else { ram_counters.downtime_bytes +=3D bytes; } - ram_counters.transferred +=3D bytes; + stat64_add(&ram_atomic_counters.transferred, bytes); } =20 void dirty_sync_missed_zero_copy(void) @@ -725,7 +732,7 @@ void mig_throttle_counter_reset(void) =20 rs->time_last_bitmap_sync =3D qemu_clock_get_ms(QEMU_CLOCK_REALTIME); rs->num_dirty_pages_period =3D 0; - rs->bytes_xfer_prev =3D ram_counters.transferred; + rs->bytes_xfer_prev =3D stat64_get(&ram_atomic_counters.transferred); } =20 /** @@ -1085,8 +1092,9 @@ uint64_t ram_pagesize_summary(void) =20 uint64_t ram_get_total_transferred_pages(void) { - return ram_counters.normal + ram_counters.duplicate + - compression_counters.pages + xbzrle_counters.pages; + return stat64_get(&ram_atomic_counters.normal) + + stat64_get(&ram_atomic_counters.duplicate) + + compression_counters.pages + xbzrle_counters.pages; } =20 static void migration_update_rates(RAMState *rs, int64_t end_time) @@ -1145,8 +1153,8 @@ static void migration_trigger_throttle(RAMState *rs) { MigrationState *s =3D migrate_get_current(); uint64_t threshold =3D s->parameters.throttle_trigger_threshold; - - uint64_t bytes_xfer_period =3D ram_counters.transferred - rs->bytes_xf= er_prev; + uint64_t bytes_xfer_period =3D + stat64_get(&ram_atomic_counters.transferred) - rs->bytes_xfer_prev; uint64_t bytes_dirty_period =3D rs->num_dirty_pages_period * TARGET_PA= GE_SIZE; uint64_t bytes_dirty_threshold =3D bytes_xfer_period * threshold / 100; =20 @@ -1209,7 +1217,7 @@ static void migration_bitmap_sync(RAMState *rs) /* reset period counters */ rs->time_last_bitmap_sync =3D end_time; rs->num_dirty_pages_period =3D 0; - rs->bytes_xfer_prev =3D ram_counters.transferred; + rs->bytes_xfer_prev =3D stat64_get(&ram_atomic_counters.transferre= d); } if (migrate_use_events()) { qapi_event_send_migration_pass(ram_counters.dirty_sync_count); @@ -1285,7 +1293,7 @@ static int save_zero_page(RAMState *rs, RAMBlock *blo= ck, ram_addr_t offset) int len =3D save_zero_page_to_file(rs, rs->f, block, offset); =20 if (len) { - ram_counters.duplicate++; + stat64_add(&ram_atomic_counters.duplicate, 1); ram_transferred_add(len); return 1; } @@ -1322,9 +1330,9 @@ static bool control_save_page(RAMState *rs, RAMBlock = *block, ram_addr_t offset, } =20 if (bytes_xmit > 0) { - ram_counters.normal++; + stat64_add(&ram_atomic_counters.normal, 1); } else if (bytes_xmit =3D=3D 0) { - ram_counters.duplicate++; + stat64_add(&ram_atomic_counters.duplicate, 1); } =20 return true; @@ -1354,7 +1362,7 @@ static int save_normal_page(RAMState *rs, RAMBlock *b= lock, ram_addr_t offset, qemu_put_buffer(rs->f, buf, TARGET_PAGE_SIZE); } ram_transferred_add(TARGET_PAGE_SIZE); - ram_counters.normal++; + stat64_add(&ram_atomic_counters.normal, 1); return 1; } =20 @@ -1410,7 +1418,7 @@ static int ram_save_multifd_page(RAMState *rs, RAMBlo= ck *block, if (multifd_queue_page(rs->f, block, offset) < 0) { return -1; } - ram_counters.normal++; + stat64_add(&ram_atomic_counters.normal, 1); =20 return 1; } @@ -1448,7 +1456,7 @@ update_compress_thread_counts(const CompressParam *pa= ram, int bytes_xmit) ram_transferred_add(bytes_xmit); =20 if (param->zero_page) { - ram_counters.duplicate++; + stat64_add(&ram_atomic_counters.duplicate, 1); return; } =20 @@ -2618,9 +2626,9 @@ void acct_update_position(QEMUFile *f, size_t size, b= ool zero) uint64_t pages =3D size / TARGET_PAGE_SIZE; =20 if (zero) { - ram_counters.duplicate +=3D pages; + stat64_add(&ram_atomic_counters.duplicate, pages); } else { - ram_counters.normal +=3D pages; + stat64_add(&ram_atomic_counters.normal, pages); ram_transferred_add(size); qemu_file_credit_transfer(f, size); } diff --git a/migration/ram.h b/migration/ram.h index c7af65ac74..9331ef5e04 100644 --- a/migration/ram.h +++ b/migration/ram.h @@ -32,7 +32,27 @@ #include "qapi/qapi-types-migration.h" #include "exec/cpu-common.h" #include "io/channel.h" +#include "qemu/stats64.h" =20 +/* + * These are the migration statistic counters that need to be updated using + * atomic ops (can be accessed by more than one thread). Here since we + * cannot modify MigrationStats directly to use Stat64 as it was defined in + * the QAPI scheme, we define an internal structure to hold them, and we + * propagate the real values when QMP queries happen. + * + * IOW, the corresponding fields within ram_counters on these specific + * fields will be always zero and not being used at all; they're just + * placeholders to make it QAPI-compatible. + */ +typedef struct { + Stat64 transferred; + Stat64 duplicate; + Stat64 normal; + Stat64 postcopy_bytes; +} MigrationAtomicStats; + +extern MigrationAtomicStats ram_atomic_counters; extern MigrationStats ram_counters; extern XBZRLECacheStats xbzrle_counters; extern CompressionStats compression_counters; --=20 2.37.3 From nobody Sun May 19 09:23:40 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1665526357; cv=none; d=zohomail.com; s=zohoarc; b=GYTN7e+cs2W9BQjylTwKXhVlbin2SSSEwuxXM8ELEh8J38UUKKCzmiHUUJuCMoYI9xU+nB5G53YhJYNeQ5U91qELhrp3cV0lHaOwH22OH4LyCIS2FT1KDt8cdas2Cp5TuW4TZXY2pImerOMt7bK7NNz48UGUD8ZbapItifgixK4= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1665526357; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=kR9egZ9JBYzulwkdfsWySbQN0WL+vswze1lfOjbCC5k=; b=UCm5DG4LUsQwDmfcRP+nGnOlyXKYbgXrsc+anOyCumHr9+7/aw/ibSVqq458deVgecX+VGauDkkgKn+0R4SxOG2x9s3CSHVajeu6e/bM5xkmc22mBAhO1qSm9AEWXPXQZ5uVXlU8IuB3L6GQaFb80MDHO66AT+A8OGsiaeGXQF8= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1665526357709375.73293459352647; Tue, 11 Oct 2022 15:12:37 -0700 (PDT) Received: from localhost ([::1]:54008 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1oiNUK-0002sX-L3 for importer@patchew.org; Tue, 11 Oct 2022 18:12:36 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:53874) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1oiNEd-0001nE-1Q for qemu-devel@nongnu.org; Tue, 11 Oct 2022 17:56:23 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:45636) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1oiNEa-0003XB-4H for qemu-devel@nongnu.org; Tue, 11 Oct 2022 17:56:22 -0400 Received: from mail-qv1-f72.google.com (mail-qv1-f72.google.com [209.85.219.72]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_128_GCM_SHA256) id us-mta-428-5g0k6xK8M8G68LHJtH8ahQ-1; Tue, 11 Oct 2022 17:56:13 -0400 Received: by mail-qv1-f72.google.com with SMTP id y2-20020a0ce802000000b004b1ce1c4a70so8699142qvn.9 for ; Tue, 11 Oct 2022 14:56:13 -0700 (PDT) Received: from x1n.redhat.com (bras-base-aurron9127w-grc-46-70-31-27-79.dsl.bell.ca. [70.31.27.79]) by smtp.gmail.com with ESMTPSA id u10-20020a05620a430a00b006ce7cd81359sm13863230qko.110.2022.10.11.14.56.11 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 11 Oct 2022 14:56:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1665525379; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=kR9egZ9JBYzulwkdfsWySbQN0WL+vswze1lfOjbCC5k=; b=LFqFZu7nAXEK46iYmBIu64qvE4NVXkHWcAZCltg0CS9i7CziDQl8mvIjxaFgMWUa7J4+BH gWCJ/hQhupXhE5aSY+pYbC1uN0lw8KkKpZjuEYHm46NaJNOMlUQzbz4SlgEuE1uOpeAu6L ZN1hgwC75Hbms6kGZZ/WVlSICbkXqbc= X-MC-Unique: 5g0k6xK8M8G68LHJtH8ahQ-1 X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=kR9egZ9JBYzulwkdfsWySbQN0WL+vswze1lfOjbCC5k=; b=kgEcsPUhb/O4Ru5vUJDg3QLfS1L2QRBJfCtMDmBkxkNaXrsStIaz1P7coHhqUQUh5z Yl5EmN7EV++0HQOlmTRd54A7ZMrGVgCQDUp+ddkzcc+MJG9Wuie7wFfWGIwn5o/omjIo i1xKS03w6ax3o0GjDkmYEJK5u/0jn+VVfSJ7y3BYxXXDXXlJ5cXjtT9b9Nj1O6rvmodv zgYFeEsOgDgs/rt+zNRpPjaKxatvtfbSZu0nF94g9sEDWfxRb2frySWH+gZfQG6VRbZb ThPtgHIKreL9GTgiq3PRjl+MyLHjXZ4/nEjdSiXicfq12Xz8cKhl86tuGcOFR9+1KXbN 4lkg== X-Gm-Message-State: ACrzQf1jIn0vaemm3yOWzQgDLVv0zxGkC8DjnVgBaWxgQ1dnIacn5Gcj t0/dWhnkXznHkqMLelBYkg/bJzC8hkVzSeX3R/JgLdVHPaBQWwOsKnX+lZe3GphWynoFc8BXQVz Zt1tZoD8qkhWd/gNOyHgizc+yyAwunOmUa+F6rQD09IjA/GHcdgTsT1bTsoXFsdot X-Received: by 2002:a05:622a:181:b0:39c:bd89:7b9 with SMTP id s1-20020a05622a018100b0039cbd8907b9mr862575qtw.590.1665525372438; Tue, 11 Oct 2022 14:56:12 -0700 (PDT) X-Google-Smtp-Source: AMsMyM7uNwwMiFHccYo2+OzyDnw34OwNbloFELgao6KY0T5cgKqn73qGzWA2WtWr94V5my/3Kah+Fg== X-Received: by 2002:a05:622a:181:b0:39c:bd89:7b9 with SMTP id s1-20020a05622a018100b0039cbd8907b9mr862550qtw.590.1665525372091; Tue, 11 Oct 2022 14:56:12 -0700 (PDT) From: Peter Xu To: qemu-devel@nongnu.org Cc: "Dr . David Alan Gilbert" , peterx@redhat.com, "Daniel P . Berrange" , Juan Quintela , ani@anisinha.ca, Leonardo Bras Soares Passos , Manish Mishra Subject: [PATCH v2 08/15] migration: Teach PSS about host page Date: Tue, 11 Oct 2022 17:55:52 -0400 Message-Id: <20221011215559.602584-9-peterx@redhat.com> X-Mailer: git-send-email 2.37.3 In-Reply-To: <20221011215559.602584-1-peterx@redhat.com> References: <20221011215559.602584-1-peterx@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=peterx@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1665526358765100001 Content-Type: text/plain; charset="utf-8" Migration code has a lot to do with host pages. Teaching PSS core about the idea of host page helps a lot and makes the code clean. Meanwhile, this prepares for the future changes that can leverage the new PSS helpers that this patch introduces to send host page in another thread. Three more fields are introduced for this: (1) host_page_sending: this is set to true when QEMU is sending a host page, false otherwise. (2) host_page_{start|end}: these point to the start/end of host page we're sending, and it's only valid when host_page_sending=3D=3Dtrue. For example, when we look up the next dirty page on the ramblock, with host_page_sending=3D=3Dtrue, we'll not try to look for anything beyond the current host page boundary. This can be slightly efficient than current code because currently we'll set pss->page to next dirty bit (which can be over current host page boundary) and reset it to host page boundary if we found it goes beyond that. With above, we can easily make migration_bitmap_find_dirty() self contained by updating pss->page properly. rs* parameter is removed because it's not even used in old code. When sending a host page, we should use the pss helpers like this: - pss_host_page_prepare(pss): called before sending host page - pss_within_range(pss): whether we're still working on the cur host page? - pss_host_page_finish(pss): called after sending a host page Then we can use ram_save_target_page() to save one small page. Currently ram_save_host_page() is still the only user. If there'll be another function to send host page (e.g. in return path thread) in the future, it should follow the same style. Reviewed-by: Dr. David Alan Gilbert Signed-off-by: Peter Xu --- migration/ram.c | 95 +++++++++++++++++++++++++++++++++++++++---------- 1 file changed, 76 insertions(+), 19 deletions(-) diff --git a/migration/ram.c b/migration/ram.c index f5a86265c7..ebb4737deb 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -481,6 +481,11 @@ struct PageSearchStatus { * postcopy pages via postcopy preempt channel. */ bool postcopy_target_channel; + /* Whether we're sending a host page */ + bool host_page_sending; + /* The start/end of current host page. Only valid if host_page_sendin= g=3D=3Dtrue */ + unsigned long host_page_start; + unsigned long host_page_end; }; typedef struct PageSearchStatus PageSearchStatus; =20 @@ -858,26 +863,38 @@ static int save_xbzrle_page(RAMState *rs, uint8_t **c= urrent_data, } =20 /** - * migration_bitmap_find_dirty: find the next dirty page from start + * pss_find_next_dirty: find the next dirty page of current ramblock * - * Returns the page offset within memory region of the start of a dirty pa= ge + * This function updates pss->page to point to the next dirty page index + * within the ramblock to migrate, or the end of ramblock when nothing + * found. Note that when pss->host_page_sending=3D=3Dtrue it means we're + * during sending a host page, so we won't look for dirty page that is + * outside the host page boundary. * - * @rs: current RAM state - * @rb: RAMBlock where to search for dirty pages - * @start: page where we start the search + * @pss: the current page search status */ -static inline -unsigned long migration_bitmap_find_dirty(RAMState *rs, RAMBlock *rb, - unsigned long start) +static void pss_find_next_dirty(PageSearchStatus *pss) { + RAMBlock *rb =3D pss->block; unsigned long size =3D rb->used_length >> TARGET_PAGE_BITS; unsigned long *bitmap =3D rb->bmap; =20 if (ramblock_is_ignored(rb)) { - return size; + /* Points directly to the end, so we know no dirty page */ + pss->page =3D size; + return; + } + + /* + * If during sending a host page, only look for dirty pages within the + * current host page being send. + */ + if (pss->host_page_sending) { + assert(pss->host_page_end); + size =3D MIN(size, pss->host_page_end); } =20 - return find_next_bit(bitmap, size, start); + pss->page =3D find_next_bit(bitmap, size, pss->page); } =20 static void migration_clear_memory_region_dirty_bitmap(RAMBlock *rb, @@ -1563,7 +1580,9 @@ static bool find_dirty_block(RAMState *rs, PageSearch= Status *pss, bool *again) pss->postcopy_requested =3D false; pss->postcopy_target_channel =3D RAM_CHANNEL_PRECOPY; =20 - pss->page =3D migration_bitmap_find_dirty(rs, pss->block, pss->page); + /* Update pss->page for the next dirty bit in ramblock */ + pss_find_next_dirty(pss); + if (pss->complete_round && pss->block =3D=3D rs->last_seen_block && pss->page >=3D rs->last_page) { /* @@ -2452,6 +2471,44 @@ static void postcopy_preempt_reset_channel(RAMState = *rs) } } =20 +/* Should be called before sending a host page */ +static void pss_host_page_prepare(PageSearchStatus *pss) +{ + /* How many guest pages are there in one host page? */ + size_t guest_pfns =3D qemu_ram_pagesize(pss->block) >> TARGET_PAGE_BIT= S; + + pss->host_page_sending =3D true; + pss->host_page_start =3D ROUND_DOWN(pss->page, guest_pfns); + pss->host_page_end =3D ROUND_UP(pss->page + 1, guest_pfns); +} + +/* + * Whether the page pointed by PSS is within the host page being sent. + * Must be called after a previous pss_host_page_prepare(). + */ +static bool pss_within_range(PageSearchStatus *pss) +{ + ram_addr_t ram_addr; + + assert(pss->host_page_sending); + + /* Over host-page boundary? */ + if (pss->page >=3D pss->host_page_end) { + return false; + } + + ram_addr =3D ((ram_addr_t)pss->page) << TARGET_PAGE_BITS; + + return offset_in_ramblock(pss->block, ram_addr); +} + +static void pss_host_page_finish(PageSearchStatus *pss) +{ + pss->host_page_sending =3D false; + /* This is not needed, but just to reset it */ + pss->host_page_start =3D pss->host_page_end =3D 0; +} + /** * ram_save_host_page: save a whole host page * @@ -2474,8 +2531,6 @@ static int ram_save_host_page(RAMState *rs, PageSearc= hStatus *pss) int tmppages, pages =3D 0; size_t pagesize_bits =3D qemu_ram_pagesize(pss->block) >> TARGET_PAGE_BITS; - unsigned long hostpage_boundary =3D - QEMU_ALIGN_UP(pss->page + 1, pagesize_bits); unsigned long start_page =3D pss->page; int res; =20 @@ -2488,6 +2543,9 @@ static int ram_save_host_page(RAMState *rs, PageSearc= hStatus *pss) postcopy_preempt_choose_channel(rs, pss); } =20 + /* Update host page boundary information */ + pss_host_page_prepare(pss); + do { if (postcopy_needs_preempt(rs, pss)) { postcopy_do_preempt(rs, pss); @@ -2525,15 +2583,14 @@ static int ram_save_host_page(RAMState *rs, PageSea= rchStatus *pss) } =20 if (tmppages < 0) { + pss_host_page_finish(pss); return tmppages; } =20 - pss->page =3D migration_bitmap_find_dirty(rs, pss->block, pss->pag= e); - } while ((pss->page < hostpage_boundary) && - offset_in_ramblock(pss->block, - ((ram_addr_t)pss->page) << TARGET_PAGE_BIT= S)); - /* The offset we leave with is the min boundary of host page and block= */ - pss->page =3D MIN(pss->page, hostpage_boundary); + pss_find_next_dirty(pss); + } while (pss_within_range(pss)); + + pss_host_page_finish(pss); =20 /* * When with postcopy preempt mode, flush the data as soon as possible= for --=20 2.37.3 From nobody Sun May 19 09:23:40 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1665526143; cv=none; d=zohomail.com; s=zohoarc; b=RpDcBYY1192DHfryAuYl4lc0Nh5bdvqcgDcXyBIEu4FGOqnn/a1hWV7LKYH4pYvQLU4azESyPn/aYKJsMJf+0HfPKCNZVni7MjuRFmGZzS7JoCyE9OZiPONDhHZxMr+yW0kXHSKgv+a3Ip7968j2az/UOVAg8vi7Wx2dIVHiv8c= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1665526143; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=Bbh+5cOdAgAg8ohxLZtNxbjBjhrO6S04LcCKSTi0LvM=; b=H2yLjIVoB3/yfUj3QcdNeqtaIbMXDS5v8J6QXaSAIjKdGmxWaJKpIDC8eFIvNzQtOVfa3bzxUehAGrgmkGxf9b9YvMhErpK0k2WB5xWapcAP/TdX7g8CK6Sn8Vx9vkcYwxAbFNJDG0St3ANMxsAOwkSabMbP2sKrrDKJeN+AjrQ= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1665526143975776.8080300443787; Tue, 11 Oct 2022 15:09:03 -0700 (PDT) Received: from localhost ([::1]:43494 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1oiNQs-0005bX-TU for importer@patchew.org; Tue, 11 Oct 2022 18:09:02 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:53866) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1oiNEY-0001if-Ee for qemu-devel@nongnu.org; Tue, 11 Oct 2022 17:56:18 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]:29543) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1oiNEW-0003We-E8 for qemu-devel@nongnu.org; Tue, 11 Oct 2022 17:56:18 -0400 Received: from mail-qk1-f200.google.com (mail-qk1-f200.google.com [209.85.222.200]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_128_GCM_SHA256) id us-mta-281-WnczVfoCNj-MM0r3mmLRfw-1; Tue, 11 Oct 2022 17:56:14 -0400 Received: by mail-qk1-f200.google.com with SMTP id bq17-20020a05620a469100b006d2c613ebcbso12784610qkb.19 for ; Tue, 11 Oct 2022 14:56:14 -0700 (PDT) Received: from x1n.redhat.com (bras-base-aurron9127w-grc-46-70-31-27-79.dsl.bell.ca. [70.31.27.79]) by smtp.gmail.com with ESMTPSA id u10-20020a05620a430a00b006ce7cd81359sm13863230qko.110.2022.10.11.14.56.12 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 11 Oct 2022 14:56:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1665525375; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Bbh+5cOdAgAg8ohxLZtNxbjBjhrO6S04LcCKSTi0LvM=; b=IdfIvC6Q1RqlPR1EBRwLDVEUXPKAI8Gx++VhEVm5xlLtRFeIqUanlKBh8Da593P20/FJ80 uBV1eIOe8RfFVP3i7pqx1DOAm0kSLo7DMBNBtNOKsEEcJIYQJ0Uu6XKEpP2Q3G9Exk4356 1uPlys0Szy6jmMxgH6vxDF9vT14jyMQ= X-MC-Unique: WnczVfoCNj-MM0r3mmLRfw-1 X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Bbh+5cOdAgAg8ohxLZtNxbjBjhrO6S04LcCKSTi0LvM=; b=mtTrkNtincjzr1zfo8bAyOcfBPIG1JlwgYld5/Qn0iHhEfX3iNA+/uuFlIMHjGKgq0 T+Mk3hZ/J3mZQ5aK9TJ3+h/NlfjXkUo8jTFf00qirmUV1pqwxUhFXUWOXNfxwIZJ+5V4 CKQQIOAGyUooC9RMOfnFaKnDB300IbVuJTkJ84neinNocfIMkljvjshIE740F/fcahvb id1zcMVpMykQCYG/01gdOS+JMK7cj/Jrvl6114JkFV6K3TGjpVaWuKbX5WnMJXOTtTQJ CVhvVrvt/xvAqwnH39bhcXKqV+ljdAS2JUVn919r3JAb41tW3oiDuxCHpRoVvNijLsTS 4Diw== X-Gm-Message-State: ACrzQf3ujXyDRKxYrteL74ZXC3lyRLhNse+91S/f6cx+PlKIZUaAShXH /3/WuiJW7tsv60nRZ+VgcGlZ228qknEjIpXQ0iSnaMqr8yx6aqb3v9tqqDk4ghIo5IdQrhhxc8G gPJIVoSDqM0ydgKjJnx8H3BTjFZRNbbAghE/tSgqYh/l2pgfvA4jhmvp9QxmMQj+z X-Received: by 2002:a05:622a:92:b0:35d:58bc:4a4a with SMTP id o18-20020a05622a009200b0035d58bc4a4amr21737639qtw.391.1665525373713; Tue, 11 Oct 2022 14:56:13 -0700 (PDT) X-Google-Smtp-Source: AMsMyM4YIUD/A56uDUGKjXwzbsoO9FtSPXhwDN5zO+UNDUEcNlb7LDFKmwYRHrasqz4pbBge/kGYJw== X-Received: by 2002:a05:622a:92:b0:35d:58bc:4a4a with SMTP id o18-20020a05622a009200b0035d58bc4a4amr21737624qtw.391.1665525373363; Tue, 11 Oct 2022 14:56:13 -0700 (PDT) From: Peter Xu To: qemu-devel@nongnu.org Cc: "Dr . David Alan Gilbert" , peterx@redhat.com, "Daniel P . Berrange" , Juan Quintela , ani@anisinha.ca, Leonardo Bras Soares Passos , Manish Mishra Subject: [PATCH v2 09/15] migration: Introduce pss_channel Date: Tue, 11 Oct 2022 17:55:53 -0400 Message-Id: <20221011215559.602584-10-peterx@redhat.com> X-Mailer: git-send-email 2.37.3 In-Reply-To: <20221011215559.602584-1-peterx@redhat.com> References: <20221011215559.602584-1-peterx@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=peterx@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1665526145602100001 Content-Type: text/plain; charset="utf-8" Introduce pss_channel for PageSearchStatus, define it as "the migration channel to be used to transfer this host page". We used to have rs->f, which is a mirror to MigrationState.to_dst_file. After postcopy preempt initial version, rs->f can be dynamically changed depending on which channel we want to use. But that later work still doesn't grant full concurrency of sending pages in e.g. different threads, because rs->f can either be the PRECOPY channel or POSTCOPY channel. This needs to be per-thread too. PageSearchStatus is actually a good piece of struct which we can leverage if we want to have multiple threads sending pages. Sending a single guest page may not make sense, so we make the granule to be "host page", and in the PSS structure we allow specify a QEMUFile* to migrate a specific host page. Then we open the possibility to specify different channels in different threads with different PSS structures. The PSS prefix can be slightly misleading here because e.g. for the upcoming usage of postcopy channel/thread it's not "searching" (or, scanning) at all but sending the explicit page that was requested. However since PSS existed for some years keep it as-is until someone complains. This patch mostly (simply) replace rs->f with pss->pss_channel only. No functional change intended for this patch yet. But it does prepare to finally drop rs->f, and make ram_save_guest_page() thread safe. Reviewed-by: Dr. David Alan Gilbert Signed-off-by: Peter Xu --- migration/ram.c | 70 +++++++++++++++++++++++++++---------------------- 1 file changed, 38 insertions(+), 32 deletions(-) diff --git a/migration/ram.c b/migration/ram.c index ebb4737deb..d81dd3fdac 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -453,6 +453,8 @@ void dirty_sync_missed_zero_copy(void) =20 /* used by the search for pages to send */ struct PageSearchStatus { + /* The migration channel used for a specific host page */ + QEMUFile *pss_channel; /* Current block being searched */ RAMBlock *block; /* Current page to search from */ @@ -775,9 +777,9 @@ static void xbzrle_cache_zero_page(RAMState *rs, ram_ad= dr_t current_addr) * @block: block that contains the page we want to send * @offset: offset inside the block for the page */ -static int save_xbzrle_page(RAMState *rs, uint8_t **current_data, - ram_addr_t current_addr, RAMBlock *block, - ram_addr_t offset) +static int save_xbzrle_page(RAMState *rs, QEMUFile *file, + uint8_t **current_data, ram_addr_t current_add= r, + RAMBlock *block, ram_addr_t offset) { int encoded_len =3D 0, bytes_xbzrle; uint8_t *prev_cached_page; @@ -845,11 +847,11 @@ static int save_xbzrle_page(RAMState *rs, uint8_t **c= urrent_data, } =20 /* Send XBZRLE based compressed page */ - bytes_xbzrle =3D save_page_header(rs, rs->f, block, + bytes_xbzrle =3D save_page_header(rs, file, block, offset | RAM_SAVE_FLAG_XBZRLE); - qemu_put_byte(rs->f, ENCODING_FLAG_XBZRLE); - qemu_put_be16(rs->f, encoded_len); - qemu_put_buffer(rs->f, XBZRLE.encoded_buf, encoded_len); + qemu_put_byte(file, ENCODING_FLAG_XBZRLE); + qemu_put_be16(file, encoded_len); + qemu_put_buffer(file, XBZRLE.encoded_buf, encoded_len); bytes_xbzrle +=3D encoded_len + 1 + 2; /* * Like compressed_size (please see update_compress_thread_counts), @@ -1305,9 +1307,10 @@ static int save_zero_page_to_file(RAMState *rs, QEMU= File *file, * @block: block that contains the page we want to send * @offset: offset inside the block for the page */ -static int save_zero_page(RAMState *rs, RAMBlock *block, ram_addr_t offset) +static int save_zero_page(RAMState *rs, QEMUFile *file, RAMBlock *block, + ram_addr_t offset) { - int len =3D save_zero_page_to_file(rs, rs->f, block, offset); + int len =3D save_zero_page_to_file(rs, file, block, offset); =20 if (len) { stat64_add(&ram_atomic_counters.duplicate, 1); @@ -1324,15 +1327,15 @@ static int save_zero_page(RAMState *rs, RAMBlock *b= lock, ram_addr_t offset) * * Return true if the pages has been saved, otherwise false is returned. */ -static bool control_save_page(RAMState *rs, RAMBlock *block, ram_addr_t of= fset, - int *pages) +static bool control_save_page(PageSearchStatus *pss, RAMBlock *block, + ram_addr_t offset, int *pages) { uint64_t bytes_xmit =3D 0; int ret; =20 *pages =3D -1; - ret =3D ram_control_save_page(rs->f, block->offset, offset, TARGET_PAG= E_SIZE, - &bytes_xmit); + ret =3D ram_control_save_page(pss->pss_channel, block->offset, offset, + TARGET_PAGE_SIZE, &bytes_xmit); if (ret =3D=3D RAM_SAVE_CONTROL_NOT_SUPP) { return false; } @@ -1366,17 +1369,17 @@ static bool control_save_page(RAMState *rs, RAMBloc= k *block, ram_addr_t offset, * @buf: the page to be sent * @async: send to page asyncly */ -static int save_normal_page(RAMState *rs, RAMBlock *block, ram_addr_t offs= et, - uint8_t *buf, bool async) +static int save_normal_page(RAMState *rs, QEMUFile *file, RAMBlock *block, + ram_addr_t offset, uint8_t *buf, bool async) { - ram_transferred_add(save_page_header(rs, rs->f, block, + ram_transferred_add(save_page_header(rs, file, block, offset | RAM_SAVE_FLAG_PAGE)); if (async) { - qemu_put_buffer_async(rs->f, buf, TARGET_PAGE_SIZE, + qemu_put_buffer_async(file, buf, TARGET_PAGE_SIZE, migrate_release_ram() && migration_in_postcopy()); } else { - qemu_put_buffer(rs->f, buf, TARGET_PAGE_SIZE); + qemu_put_buffer(file, buf, TARGET_PAGE_SIZE); } ram_transferred_add(TARGET_PAGE_SIZE); stat64_add(&ram_atomic_counters.normal, 1); @@ -1409,8 +1412,8 @@ static int ram_save_page(RAMState *rs, PageSearchStat= us *pss) =20 XBZRLE_cache_lock(); if (rs->xbzrle_enabled && !migration_in_postcopy()) { - pages =3D save_xbzrle_page(rs, &p, current_addr, block, - offset); + pages =3D save_xbzrle_page(rs, pss->pss_channel, &p, current_addr, + block, offset); if (!rs->last_stage) { /* Can't send this cached data async, since the cache page * might get updated before it gets to the wire @@ -1421,7 +1424,8 @@ static int ram_save_page(RAMState *rs, PageSearchStat= us *pss) =20 /* XBZRLE overflow or normal page */ if (pages =3D=3D -1) { - pages =3D save_normal_page(rs, block, offset, p, send_async); + pages =3D save_normal_page(rs, pss->pss_channel, block, offset, + p, send_async); } =20 XBZRLE_cache_unlock(); @@ -1429,10 +1433,10 @@ static int ram_save_page(RAMState *rs, PageSearchSt= atus *pss) return pages; } =20 -static int ram_save_multifd_page(RAMState *rs, RAMBlock *block, +static int ram_save_multifd_page(QEMUFile *file, RAMBlock *block, ram_addr_t offset) { - if (multifd_queue_page(rs->f, block, offset) < 0) { + if (multifd_queue_page(file, block, offset) < 0) { return -1; } stat64_add(&ram_atomic_counters.normal, 1); @@ -1727,7 +1731,7 @@ static int ram_save_release_protection(RAMState *rs, = PageSearchStatus *pss, uint64_t run_length =3D (pss->page - start_page) << TARGET_PAGE_BI= TS; =20 /* Flush async buffers before un-protect. */ - qemu_fflush(rs->f); + qemu_fflush(pss->pss_channel); /* Un-protect memory range. */ res =3D uffd_change_protection(rs->uffdio_fd, page_address, run_le= ngth, false, false); @@ -2314,7 +2318,7 @@ static int ram_save_target_page(RAMState *rs, PageSea= rchStatus *pss) ram_addr_t offset =3D ((ram_addr_t)pss->page) << TARGET_PAGE_BITS; int res; =20 - if (control_save_page(rs, block, offset, &res)) { + if (control_save_page(pss, block, offset, &res)) { return res; } =20 @@ -2322,7 +2326,7 @@ static int ram_save_target_page(RAMState *rs, PageSea= rchStatus *pss) return 1; } =20 - res =3D save_zero_page(rs, block, offset); + res =3D save_zero_page(rs, pss->pss_channel, block, offset); if (res > 0) { /* Must let xbzrle know, otherwise a previous (now 0'd) cached * page would be stale @@ -2342,7 +2346,7 @@ static int ram_save_target_page(RAMState *rs, PageSea= rchStatus *pss) * still see partially copied pages which is data corruption. */ if (migrate_use_multifd() && !migration_in_postcopy()) { - return ram_save_multifd_page(rs, block, offset); + return ram_save_multifd_page(pss->pss_channel, block, offset); } =20 return ram_save_page(rs, pss); @@ -2539,10 +2543,6 @@ static int ram_save_host_page(RAMState *rs, PageSear= chStatus *pss) return 0; } =20 - if (postcopy_preempt_active()) { - postcopy_preempt_choose_channel(rs, pss); - } - /* Update host page boundary information */ pss_host_page_prepare(pss); =20 @@ -2602,7 +2602,7 @@ static int ram_save_host_page(RAMState *rs, PageSearc= hStatus *pss) * explicit flush or it won't flush until the buffer is full. */ if (migrate_postcopy_preempt() && pss->postcopy_requested) { - qemu_fflush(rs->f); + qemu_fflush(pss->pss_channel); } =20 res =3D ram_save_release_protection(rs, pss, start_page); @@ -2668,6 +2668,12 @@ static int ram_find_and_save_block(RAMState *rs) } =20 if (found) { + /* Update rs->f with correct channel */ + if (postcopy_preempt_active()) { + postcopy_preempt_choose_channel(rs, &pss); + } + /* Cache rs->f in pss_channel (TODO: remove rs->f) */ + pss.pss_channel =3D rs->f; pages =3D ram_save_host_page(rs, &pss); } } while (!pages && again); --=20 2.37.3 From nobody Sun May 19 09:23:40 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1665526566; cv=none; d=zohomail.com; s=zohoarc; b=LxaaTysT+M0ahz+iUlfci2jwdH+N8qbG/aGiBp3YeklPIxa5ozpwfn9z8OG/S9deS1/xZy4TdACBBa8qPsuNk2tQqiKCAWGt8ifSunN9vtZeNtK/NANPwh6O8TLLA7IvRn8/Kg0hIEP+Li0TgtdcUwESBgV4lsonEcXWkoidASg= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1665526566; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=cxfQO/4sHnW7QhOj2AN2X9RclF0rR+FjXdBj5STjEUY=; b=f/ikSnrUWWwMOBfku03ZqAJGMF6n7EwQKr4bbVguUPsFDEJnE7scUPzmYKv65vkTS7fbNTBWKIg3LcETTls17znobWW2SyhH6pRR10XRlStEHHZLvh8folAQkvEeD0RGDckyEPW1jPApIFLb1RvAig5HjNhzdCcswN2Jpz2TFZE= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1665526566043656.8627477445448; Tue, 11 Oct 2022 15:16:06 -0700 (PDT) Received: from localhost ([::1]:38754 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1oiNXg-0001Cr-VZ for importer@patchew.org; Tue, 11 Oct 2022 18:16:04 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:53868) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1oiNEY-0001ih-Rj for qemu-devel@nongnu.org; Tue, 11 Oct 2022 17:56:18 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:36314) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1oiNEX-0003Wj-9H for qemu-devel@nongnu.org; Tue, 11 Oct 2022 17:56:18 -0400 Received: from mail-qv1-f69.google.com (mail-qv1-f69.google.com [209.85.219.69]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_128_GCM_SHA256) id us-mta-436-03xybeSzObmwA7bFB21DDw-1; Tue, 11 Oct 2022 17:56:15 -0400 Received: by mail-qv1-f69.google.com with SMTP id dn14-20020a056214094e00b004b1a231394eso8788514qvb.13 for ; Tue, 11 Oct 2022 14:56:15 -0700 (PDT) Received: from x1n.redhat.com (bras-base-aurron9127w-grc-46-70-31-27-79.dsl.bell.ca. [70.31.27.79]) by smtp.gmail.com with ESMTPSA id u10-20020a05620a430a00b006ce7cd81359sm13863230qko.110.2022.10.11.14.56.13 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 11 Oct 2022 14:56:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1665525376; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=cxfQO/4sHnW7QhOj2AN2X9RclF0rR+FjXdBj5STjEUY=; b=EC2KleJmS6vfgFQvBcPEAEfNWxaZtEeqcs8bBVrtpSiskmuoj6jbwzLoOsrYcEpYTxVHm9 m/Y3qm0N1YGTmwJIXDnf72UmN0DtlnuXGwd3CvAFDAz5dusHI7AP7MvKEkKU3GbxnBwuzI vqv1Iio3d0i1FphZEahySPUwnBuzUHY= X-MC-Unique: 03xybeSzObmwA7bFB21DDw-1 X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=cxfQO/4sHnW7QhOj2AN2X9RclF0rR+FjXdBj5STjEUY=; b=TDyhXbVGtXh23489Tc2K9mHNpCcMRGi+EnpYdni7WCggQ3aXJoPfX8gePIqjeFIiv7 GdwiN/oPMNG8n6B+Q5sqhfPyDwDIzSRihq5Bz1xhvHWz8wJrggB5ajze29T6L0Gd0ZUO PEo/c0zpsEObhSo1fnhZfblmUTQhfxqWou0nKqSMLOFpTXQ7tF2Tn7jIYKo4AunU0d4A Bps+tn2619pQGCnvn2Yr4pu8BZLoT6SLuuW88YrjSeY8FCiKkgEPAF0k9akx2JuwKx2k D6H/v7t1NQej/7v2JDBofCQu+FXXLOoMRLSjcVDXW+WPfgxZ/eanUSUW23VZ8FAWNIG9 qJTA== X-Gm-Message-State: ACrzQf194l69S27VHJ2I/JzlXL7i7+FSFoUoS+jm4bKyqvQIthVi4gJw JHTZZ47BfBe/Dfhxyq1sQKabPM5oErK1YsCbzhPCFsTzYwztmbduR2cudnZFCYWzJzL7WADHFeP I16/yROp6zyWeutj7PHNIvtbRx6Mg+fS4W3Jrfi7aqTDvRKi6o/T6OLDUMOl19FtG X-Received: by 2002:a05:620a:204d:b0:6ec:7d7:c267 with SMTP id d13-20020a05620a204d00b006ec07d7c267mr11751157qka.590.1665525375037; Tue, 11 Oct 2022 14:56:15 -0700 (PDT) X-Google-Smtp-Source: AMsMyM6bm54jY9dgdD8cOxq/0EAEq85cYFuKp45jPlHzhIZiADhr+nuwEdGWnCeAks+dq+LubWJppw== X-Received: by 2002:a05:620a:204d:b0:6ec:7d7:c267 with SMTP id d13-20020a05620a204d00b006ec07d7c267mr11751140qka.590.1665525374768; Tue, 11 Oct 2022 14:56:14 -0700 (PDT) From: Peter Xu To: qemu-devel@nongnu.org Cc: "Dr . David Alan Gilbert" , peterx@redhat.com, "Daniel P . Berrange" , Juan Quintela , ani@anisinha.ca, Leonardo Bras Soares Passos , Manish Mishra Subject: [PATCH v2 10/15] migration: Add pss_init() Date: Tue, 11 Oct 2022 17:55:54 -0400 Message-Id: <20221011215559.602584-11-peterx@redhat.com> X-Mailer: git-send-email 2.37.3 In-Reply-To: <20221011215559.602584-1-peterx@redhat.com> References: <20221011215559.602584-1-peterx@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=peterx@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1665526567123100001 Content-Type: text/plain; charset="utf-8" Helper to init PSS structures. Reviewed-by: Dr. David Alan Gilbert Signed-off-by: Peter Xu --- migration/ram.c | 12 +++++++++--- 1 file changed, 9 insertions(+), 3 deletions(-) diff --git a/migration/ram.c b/migration/ram.c index d81dd3fdac..44967e72b2 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -542,6 +542,14 @@ static bool do_compress_ram_page(QEMUFile *f, z_stream= *stream, RAMBlock *block, static void postcopy_preempt_restore(RAMState *rs, PageSearchStatus *pss, bool postcopy_requested); =20 +/* NOTE: page is the PFN not real ram_addr_t. */ +static void pss_init(PageSearchStatus *pss, RAMBlock *rb, ram_addr_t page) +{ + pss->block =3D rb; + pss->page =3D page; + pss->complete_round =3D false; +} + static void *do_data_compress(void *opaque) { CompressParam *param =3D opaque; @@ -2645,9 +2653,7 @@ static int ram_find_and_save_block(RAMState *rs) rs->last_page =3D 0; } =20 - pss.block =3D rs->last_seen_block; - pss.page =3D rs->last_page; - pss.complete_round =3D false; + pss_init(&pss, rs->last_seen_block, rs->last_page); =20 do { again =3D true; --=20 2.37.3 From nobody Sun May 19 09:23:40 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1665526077; cv=none; d=zohomail.com; s=zohoarc; b=KSA6mbXSkdjM9SAvRw7kEMfbF8FezqQJ1oMIuxWBQJ4YpDaaBFyyk2gcDhLsQlJsbsrTVV4FYkxD8BhUMR8MuFosi9Xam230v9NsJUnBbEWyNJeKMhNkX+JsRmOITMF7aRWqNcqyuC+/Ga0PWfqbsUYlpWNEtU1WC1p4STMnupo= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1665526077; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=tat2JQZLD/IPyTBFrKEfM41kbJ+PcH6ibpRcVlMaUSA=; b=XeCuLTpuoZPbQbrASZug23fg03P32birVpMO3XsDDDlGsACNNPnNKzYDV4R72E5YxbtSfBO6HXSZSvzXceELNOo3srDY/yvAUv4rIlU+TVztyUIZy7QnV5cwXHEGuL79+Lf8kBVaWO4RfNsi17HLS6Glmy2rLRDrLi0WE6O2868= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1665526076989395.98831002366944; Tue, 11 Oct 2022 15:07:56 -0700 (PDT) Received: from localhost ([::1]:48852 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1oiNPm-0003lL-VS for importer@patchew.org; Tue, 11 Oct 2022 18:07:55 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:53870) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1oiNEb-0001kl-Ah for qemu-devel@nongnu.org; Tue, 11 Oct 2022 17:56:21 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]:39553) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1oiNEY-0003Wu-Ql for qemu-devel@nongnu.org; Tue, 11 Oct 2022 17:56:20 -0400 Received: from mail-qv1-f70.google.com (mail-qv1-f70.google.com [209.85.219.70]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_128_GCM_SHA256) id us-mta-639-uU-FAt2SMPG9J3ukrC__CA-1; Tue, 11 Oct 2022 17:56:17 -0400 Received: by mail-qv1-f70.google.com with SMTP id jn13-20020ad45ded000000b004b1d055fbc7so8685578qvb.2 for ; Tue, 11 Oct 2022 14:56:17 -0700 (PDT) Received: from x1n.redhat.com (bras-base-aurron9127w-grc-46-70-31-27-79.dsl.bell.ca. [70.31.27.79]) by smtp.gmail.com with ESMTPSA id u10-20020a05620a430a00b006ce7cd81359sm13863230qko.110.2022.10.11.14.56.14 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 11 Oct 2022 14:56:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1665525378; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=tat2JQZLD/IPyTBFrKEfM41kbJ+PcH6ibpRcVlMaUSA=; b=WaIkDff3OgIt+l8gz0cJBcl332qDLTH4pfuOOl+4sUaGQymEK6NtWOdGdupzVQ2J++aumu 5qeJd7arVl/Zacwieo4Gt/YkZ8yZ1xa3jC5P3jfEa947NK6H+D0ddmlCe0l5L7dtVje3Vy PK1WHwkrHymxM1aBglOt+Ym5uq/4oPA= X-MC-Unique: uU-FAt2SMPG9J3ukrC__CA-1 X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=tat2JQZLD/IPyTBFrKEfM41kbJ+PcH6ibpRcVlMaUSA=; b=b/l7SAzG7sNGcPt4UGAdO/84AQfV28n71i6dERqrGfmgi7DwIOQK3CStxinycFkjsY 4V3LEgXX31MYPY9Qtu7N0Ef3f9/X+5Pi2v1p38oDFqRvFl248Gr/OIVWNUl6jCbo4Liy IL3oG+BhEVb48iFQFStai6fgWOZsb+sDrSYpotcTOONARw066MoaPQjFsWmto9OucYvF AlF/3YhAg2kkf9AR1Zif+PvyXNBPCHSw2LkZUsrRh66hJnUukyAcoKPW5omaTEjyh7Vt nWD5QcF+SqKXOI2GGL2pxQYocGhxTlSMml+DRP3H3ThagVQ3UNGvkxx7DK4KzNojfshl 0KBA== X-Gm-Message-State: ACrzQf0xL8bmOnQGwUiTeO2DOFuHbMuwlbt9b+FakTXbtlFp4GuNqrX2 RBGl/qFYh/z/L9AGvzM/uPEUREgOqKEzQ0PkYKf+MepflDgmI5vR3A3L9UKVWN76p52PWNiIx0V 1Nv/V/lSh2GwSl8SGIuoPPnCsUBkQ3AF/xoLQ7TfYWyQcub5jgivDkKGJ33L3oUCQ X-Received: by 2002:a05:620a:bc3:b0:6b6:5746:f91f with SMTP id s3-20020a05620a0bc300b006b65746f91fmr18469796qki.391.1665525376347; Tue, 11 Oct 2022 14:56:16 -0700 (PDT) X-Google-Smtp-Source: AMsMyM66W3U9u1cg19G7Zn0NTMm0NFJmCfRah0dxn/Xw5zfaes05Thsgmlg2+qM55PfrZTYmn25Wkg== X-Received: by 2002:a05:620a:bc3:b0:6b6:5746:f91f with SMTP id s3-20020a05620a0bc300b006b65746f91fmr18469777qki.391.1665525376002; Tue, 11 Oct 2022 14:56:16 -0700 (PDT) From: Peter Xu To: qemu-devel@nongnu.org Cc: "Dr . David Alan Gilbert" , peterx@redhat.com, "Daniel P . Berrange" , Juan Quintela , ani@anisinha.ca, Leonardo Bras Soares Passos , Manish Mishra Subject: [PATCH v2 11/15] migration: Make PageSearchStatus part of RAMState Date: Tue, 11 Oct 2022 17:55:55 -0400 Message-Id: <20221011215559.602584-12-peterx@redhat.com> X-Mailer: git-send-email 2.37.3 In-Reply-To: <20221011215559.602584-1-peterx@redhat.com> References: <20221011215559.602584-1-peterx@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=peterx@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1665526079200100001 Content-Type: text/plain; charset="utf-8" We used to allocate PSS structure on the stack for precopy when sending pages. Make it static, so as to describe per-channel ram migration status. Here we declared RAM_CHANNEL_MAX instances, preparing for postcopy to use it, even though this patch has not yet to start using the 2nd instance. This should not have any functional change per se, but it already starts to export PSS information via the RAMState, so that e.g. one PSS channel can start to reference the other PSS channel. Always protect PSS access using the same RAMState.bitmap_mutex. We already do so, so no code change needed, just some comment update. Maybe we should consider renaming bitmap_mutex some day as it's going to be a more commonly and big mutex we use for ram states, but just leave it for later. Reviewed-by: Dr. David Alan Gilbert Signed-off-by: Peter Xu --- migration/ram.c | 112 ++++++++++++++++++++++++++---------------------- 1 file changed, 61 insertions(+), 51 deletions(-) diff --git a/migration/ram.c b/migration/ram.c index 44967e72b2..486dc47583 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -85,6 +85,46 @@ =20 XBZRLECacheStats xbzrle_counters; =20 +/* used by the search for pages to send */ +struct PageSearchStatus { + /* The migration channel used for a specific host page */ + QEMUFile *pss_channel; + /* Current block being searched */ + RAMBlock *block; + /* Current page to search from */ + unsigned long page; + /* Set once we wrap around */ + bool complete_round; + /* + * [POSTCOPY-ONLY] Whether current page is explicitly requested by + * postcopy. When set, the request is "urgent" because the dest QEMU + * threads are waiting for us. + */ + bool postcopy_requested; + /* + * [POSTCOPY-ONLY] The target channel to use to send current page. + * + * Note: This may _not_ match with the value in postcopy_requested + * above. Let's imagine the case where the postcopy request is exactly + * the page that we're sending in progress during precopy. In this case + * we'll have postcopy_requested set to true but the target channel + * will be the precopy channel (so that we don't split brain on that + * specific page since the precopy channel already contains partial of + * that page data). + * + * Besides that specific use case, postcopy_target_channel should + * always be equal to postcopy_requested, because by default we send + * postcopy pages via postcopy preempt channel. + */ + bool postcopy_target_channel; + /* Whether we're sending a host page */ + bool host_page_sending; + /* The start/end of current host page. Invalid if host_page_sending= =3D=3Dfalse */ + unsigned long host_page_start; + unsigned long host_page_end; +}; +typedef struct PageSearchStatus PageSearchStatus; + /* struct contains XBZRLE cache and a static page used by the compression */ static struct { @@ -319,6 +359,11 @@ typedef struct { struct RAMState { /* QEMUFile used for this migration */ QEMUFile *f; + /* + * PageSearchStatus structures for the channels when send pages. + * Protected by the bitmap_mutex. + */ + PageSearchStatus pss[RAM_CHANNEL_MAX]; /* UFFD file descriptor, used in 'write-tracking' migration */ int uffdio_fd; /* Last block that we have visited searching for dirty pages */ @@ -362,7 +407,12 @@ struct RAMState { uint64_t target_page_count; /* number of dirty bits in the bitmap */ uint64_t migration_dirty_pages; - /* Protects modification of the bitmap and migration dirty pages */ + /* + * Protects: + * - dirty/clear bitmap + * - migration_dirty_pages + * - pss structures + */ QemuMutex bitmap_mutex; /* The RAMBlock used in the last src_page_requests */ RAMBlock *last_req_rb; @@ -451,46 +501,6 @@ void dirty_sync_missed_zero_copy(void) ram_counters.dirty_sync_missed_zero_copy++; } =20 -/* used by the search for pages to send */ -struct PageSearchStatus { - /* The migration channel used for a specific host page */ - QEMUFile *pss_channel; - /* Current block being searched */ - RAMBlock *block; - /* Current page to search from */ - unsigned long page; - /* Set once we wrap around */ - bool complete_round; - /* - * [POSTCOPY-ONLY] Whether current page is explicitly requested by - * postcopy. When set, the request is "urgent" because the dest QEMU - * threads are waiting for us. - */ - bool postcopy_requested; - /* - * [POSTCOPY-ONLY] The target channel to use to send current page. - * - * Note: This may _not_ match with the value in postcopy_requested - * above. Let's imagine the case where the postcopy request is exactly - * the page that we're sending in progress during precopy. In this case - * we'll have postcopy_requested set to true but the target channel - * will be the precopy channel (so that we don't split brain on that - * specific page since the precopy channel already contains partial of - * that page data). - * - * Besides that specific use case, postcopy_target_channel should - * always be equal to postcopy_requested, because by default we send - * postcopy pages via postcopy preempt channel. - */ - bool postcopy_target_channel; - /* Whether we're sending a host page */ - bool host_page_sending; - /* The start/end of current host page. Only valid if host_page_sendin= g=3D=3Dtrue */ - unsigned long host_page_start; - unsigned long host_page_end; -}; -typedef struct PageSearchStatus PageSearchStatus; - CompressionStats compression_counters; =20 struct CompressParam { @@ -2632,7 +2642,7 @@ static int ram_save_host_page(RAMState *rs, PageSearc= hStatus *pss) */ static int ram_find_and_save_block(RAMState *rs) { - PageSearchStatus pss; + PageSearchStatus *pss =3D &rs->pss[RAM_CHANNEL_PRECOPY]; int pages =3D 0; bool again, found; =20 @@ -2653,11 +2663,11 @@ static int ram_find_and_save_block(RAMState *rs) rs->last_page =3D 0; } =20 - pss_init(&pss, rs->last_seen_block, rs->last_page); + pss_init(pss, rs->last_seen_block, rs->last_page); =20 do { again =3D true; - found =3D get_queued_page(rs, &pss); + found =3D get_queued_page(rs, pss); =20 if (!found) { /* @@ -2665,27 +2675,27 @@ static int ram_find_and_save_block(RAMState *rs) * preempted precopy. Otherwise find the next dirty bit. */ if (postcopy_preempt_triggered(rs)) { - postcopy_preempt_restore(rs, &pss, false); + postcopy_preempt_restore(rs, pss, false); found =3D true; } else { /* priority queue empty, so just search for something dirt= y */ - found =3D find_dirty_block(rs, &pss, &again); + found =3D find_dirty_block(rs, pss, &again); } } =20 if (found) { /* Update rs->f with correct channel */ if (postcopy_preempt_active()) { - postcopy_preempt_choose_channel(rs, &pss); + postcopy_preempt_choose_channel(rs, pss); } /* Cache rs->f in pss_channel (TODO: remove rs->f) */ - pss.pss_channel =3D rs->f; - pages =3D ram_save_host_page(rs, &pss); + pss->pss_channel =3D rs->f; + pages =3D ram_save_host_page(rs, pss); } } while (!pages && again); =20 - rs->last_seen_block =3D pss.block; - rs->last_page =3D pss.page; + rs->last_seen_block =3D pss->block; + rs->last_page =3D pss->page; =20 return pages; } --=20 2.37.3 From nobody Sun May 19 09:23:40 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1665526414; cv=none; d=zohomail.com; s=zohoarc; b=BwP6n0G8Fdg9lHFBp2Npc8x0sYzHVE85Cz49C0ppf/h6o2RRAsWOb4jockGxith1Snz0PFSFocCHXRKeUKmhVMOFgaxDTdiYNW0jVUa+IGUWWpoziboQy6go7Iw7+Tjma2i8NLnfN47L57GePYOixXhCbMBc6CrpopXlvTB6mzI= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1665526414; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=+qNprFr/1eDDP3j4T1lBYpK9pi5Brj0EwR/Px1PnJqw=; b=SIkxCs/tbBynF4Aruz6OB5VNERlTdUd9IuLFDxw3At0YJKQtGHnQvOJJAnVZn5JzJXlsNXgiel79Rk/nKqQ6UiSUdr2LZ8GuDWCO2+gX2rFKlLDGGmlNur+xjZyyyBOCXFpEr3dVAu0JUMPGsLN7PUtkgDP7uoYwnl+ulIZ7lHI= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 166552641469530.08292325094783; Tue, 11 Oct 2022 15:13:34 -0700 (PDT) Received: from localhost ([::1]:46666 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1oiNVF-0004dp-Gn for importer@patchew.org; Tue, 11 Oct 2022 18:13:33 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:53872) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1oiNEc-0001lv-2A for qemu-devel@nongnu.org; Tue, 11 Oct 2022 17:56:22 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]:51428) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1oiNEa-0003X8-0K for qemu-devel@nongnu.org; Tue, 11 Oct 2022 17:56:21 -0400 Received: from mail-qk1-f198.google.com (mail-qk1-f198.google.com [209.85.222.198]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_128_GCM_SHA256) id us-mta-15-HP-mxFUROUC7v51PU5MuJA-1; Tue, 11 Oct 2022 17:56:18 -0400 Received: by mail-qk1-f198.google.com with SMTP id x22-20020a05620a259600b006b552a69231so12706059qko.18 for ; Tue, 11 Oct 2022 14:56:18 -0700 (PDT) Received: from x1n.redhat.com (bras-base-aurron9127w-grc-46-70-31-27-79.dsl.bell.ca. [70.31.27.79]) by smtp.gmail.com with ESMTPSA id u10-20020a05620a430a00b006ce7cd81359sm13863230qko.110.2022.10.11.14.56.16 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 11 Oct 2022 14:56:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1665525379; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=+qNprFr/1eDDP3j4T1lBYpK9pi5Brj0EwR/Px1PnJqw=; b=DdqFSR1jeSp8VQ54/9+O3DFjbijVDtU0jDYr30vOkJHlhrQQbNUTHIv24IIZCdel2UBHyG ICmY58yyx4VYDQzp0HXcDwXxv/V0FM69kWigj3v0Es0IO+d336xET3jL6KRaF/RFASLgCm cz+PRTjGuYW99inayyl/awwI3TZOHBU= X-MC-Unique: HP-mxFUROUC7v51PU5MuJA-1 X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=+qNprFr/1eDDP3j4T1lBYpK9pi5Brj0EwR/Px1PnJqw=; b=Yjr79HSEa6JiK/7wKhvNsj8LSaK1BuzX0AbXGc+tJQjPYXpWWyZZbiWg89l5LOYDaV GVZ5JNtzmRved8EwAPEoFJpgav1D3dvN8Ly35RBKdun6Qe5u82J0wAlRl5Y2bEqGeD4b 2W8TieBnoZiLhz2P8BthBCEh9+NSul+4fEOfvzdkrCfPVOzvZeN3QQnZ/9yD8ke4SYL8 C9WEpUbyB+L94dpNKPPZG52raYUfDQS0unHZ3nDk7vsTQBFjjtavOfbTjN7byz1X3Fhw gUaFZmR+NljI5Mg22KNNvG7j5LmVIELTP4S9T4Q2Ccs24ntVKPqE4SYNMmHoAiveWGgT ksuA== X-Gm-Message-State: ACrzQf0pFVvwivwjBxYX9RSLzqa37H+/0VZ01ZyxPnt1uiteOmmyxuA/ diVXQsTxP8UpRmecaDVBBEl7P8D2DyXa5Nuc7sjI5gqa5wm9OQypXJ1hTuhRCooxaCwwJz56C6+ JrW2HclQ22GOCCo9VctJHSU3nl/jSfOz0LUMh7XhRURN09f0E4tnQVxRCbrAfgO2l X-Received: by 2002:a05:620a:1706:b0:6ce:9b88:92a4 with SMTP id az6-20020a05620a170600b006ce9b8892a4mr17821553qkb.555.1665525377509; Tue, 11 Oct 2022 14:56:17 -0700 (PDT) X-Google-Smtp-Source: AMsMyM7zdlJA+P9Y7bD0Dhntin3nK7AWBEovJCVJj6s71/WARyeXUybKoLZ/5hQctECRnwTuDPzttg== X-Received: by 2002:a05:620a:1706:b0:6ce:9b88:92a4 with SMTP id az6-20020a05620a170600b006ce9b8892a4mr17821528qkb.555.1665525377134; Tue, 11 Oct 2022 14:56:17 -0700 (PDT) From: Peter Xu To: qemu-devel@nongnu.org Cc: "Dr . David Alan Gilbert" , peterx@redhat.com, "Daniel P . Berrange" , Juan Quintela , ani@anisinha.ca, Leonardo Bras Soares Passos , Manish Mishra Subject: [PATCH v2 12/15] migration: Move last_sent_block into PageSearchStatus Date: Tue, 11 Oct 2022 17:55:56 -0400 Message-Id: <20221011215559.602584-13-peterx@redhat.com> X-Mailer: git-send-email 2.37.3 In-Reply-To: <20221011215559.602584-1-peterx@redhat.com> References: <20221011215559.602584-1-peterx@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=peterx@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1665526415178100001 Content-Type: text/plain; charset="utf-8" Since we use PageSearchStatus to represent a channel, it makes perfect sense to keep last_sent_block (aka, leverage RAM_SAVE_FLAG_CONTINUE) to be per-channel rather than global because each channel can be sending different pages on ramblocks. Hence move it from RAMState into PageSearchStatus. Reviewed-by: Dr. David Alan Gilbert Signed-off-by: Peter Xu Reviewed-by: Juan Quintela --- migration/ram.c | 71 ++++++++++++++++++++++++++++--------------------- 1 file changed, 41 insertions(+), 30 deletions(-) diff --git a/migration/ram.c b/migration/ram.c index 486dc47583..cbaa5650b8 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -89,6 +89,8 @@ XBZRLECacheStats xbzrle_counters; struct PageSearchStatus { /* The migration channel used for a specific host page */ QEMUFile *pss_channel; + /* Last block from where we have sent data */ + RAMBlock *last_sent_block; /* Current block being searched */ RAMBlock *block; /* Current page to search from */ @@ -368,8 +370,6 @@ struct RAMState { int uffdio_fd; /* Last block that we have visited searching for dirty pages */ RAMBlock *last_seen_block; - /* Last block from where we have sent data */ - RAMBlock *last_sent_block; /* Last dirty target page we have sent */ ram_addr_t last_page; /* last ram version we have seen */ @@ -684,16 +684,17 @@ exit: * * Returns the number of bytes written * - * @f: QEMUFile where to send the data + * @pss: current PSS channel status * @block: block that contains the page we want to send * @offset: offset inside the block for the page * in the lower bits, it contains flags */ -static size_t save_page_header(RAMState *rs, QEMUFile *f, RAMBlock *block, +static size_t save_page_header(PageSearchStatus *pss, RAMBlock *block, ram_addr_t offset) { size_t size, len; - bool same_block =3D (block =3D=3D rs->last_sent_block); + bool same_block =3D (block =3D=3D pss->last_sent_block); + QEMUFile *f =3D pss->pss_channel; =20 if (same_block) { offset |=3D RAM_SAVE_FLAG_CONTINUE; @@ -706,7 +707,7 @@ static size_t save_page_header(RAMState *rs, QEMUFile *= f, RAMBlock *block, qemu_put_byte(f, len); qemu_put_buffer(f, (uint8_t *)block->idstr, len); size +=3D 1 + len; - rs->last_sent_block =3D block; + pss->last_sent_block =3D block; } return size; } @@ -790,17 +791,19 @@ static void xbzrle_cache_zero_page(RAMState *rs, ram_= addr_t current_addr) * -1 means that xbzrle would be longer than normal * * @rs: current RAM state + * @pss: current PSS channel * @current_data: pointer to the address of the page contents * @current_addr: addr of the page * @block: block that contains the page we want to send * @offset: offset inside the block for the page */ -static int save_xbzrle_page(RAMState *rs, QEMUFile *file, +static int save_xbzrle_page(RAMState *rs, PageSearchStatus *pss, uint8_t **current_data, ram_addr_t current_add= r, RAMBlock *block, ram_addr_t offset) { int encoded_len =3D 0, bytes_xbzrle; uint8_t *prev_cached_page; + QEMUFile *file =3D pss->pss_channel; =20 if (!cache_is_cached(XBZRLE.cache, current_addr, ram_counters.dirty_sync_count)) { @@ -865,7 +868,7 @@ static int save_xbzrle_page(RAMState *rs, QEMUFile *fil= e, } =20 /* Send XBZRLE based compressed page */ - bytes_xbzrle =3D save_page_header(rs, file, block, + bytes_xbzrle =3D save_page_header(pss, block, offset | RAM_SAVE_FLAG_XBZRLE); qemu_put_byte(file, ENCODING_FLAG_XBZRLE); qemu_put_be16(file, encoded_len); @@ -1296,19 +1299,19 @@ static void ram_release_page(const char *rbname, ui= nt64_t offset) * Returns the size of data written to the file, 0 means the page is not * a zero page * - * @rs: current RAM state - * @file: the file where the data is saved + * @pss: current PSS channel * @block: block that contains the page we want to send * @offset: offset inside the block for the page */ -static int save_zero_page_to_file(RAMState *rs, QEMUFile *file, +static int save_zero_page_to_file(PageSearchStatus *pss, RAMBlock *block, ram_addr_t offset) { uint8_t *p =3D block->host + offset; + QEMUFile *file =3D pss->pss_channel; int len =3D 0; =20 if (buffer_is_zero(p, TARGET_PAGE_SIZE)) { - len +=3D save_page_header(rs, file, block, offset | RAM_SAVE_FLAG_= ZERO); + len +=3D save_page_header(pss, block, offset | RAM_SAVE_FLAG_ZERO); qemu_put_byte(file, 0); len +=3D 1; ram_release_page(block->idstr, offset); @@ -1321,14 +1324,14 @@ static int save_zero_page_to_file(RAMState *rs, QEM= UFile *file, * * Returns the number of pages written. * - * @rs: current RAM state + * @pss: current PSS channel * @block: block that contains the page we want to send * @offset: offset inside the block for the page */ -static int save_zero_page(RAMState *rs, QEMUFile *file, RAMBlock *block, +static int save_zero_page(PageSearchStatus *pss, RAMBlock *block, ram_addr_t offset) { - int len =3D save_zero_page_to_file(rs, file, block, offset); + int len =3D save_zero_page_to_file(pss, block, offset); =20 if (len) { stat64_add(&ram_atomic_counters.duplicate, 1); @@ -1381,16 +1384,18 @@ static bool control_save_page(PageSearchStatus *pss= , RAMBlock *block, * * Returns the number of pages written. * - * @rs: current RAM state + * @pss: current PSS channel * @block: block that contains the page we want to send * @offset: offset inside the block for the page * @buf: the page to be sent * @async: send to page asyncly */ -static int save_normal_page(RAMState *rs, QEMUFile *file, RAMBlock *block, +static int save_normal_page(PageSearchStatus *pss, RAMBlock *block, ram_addr_t offset, uint8_t *buf, bool async) { - ram_transferred_add(save_page_header(rs, file, block, + QEMUFile *file =3D pss->pss_channel; + + ram_transferred_add(save_page_header(pss, block, offset | RAM_SAVE_FLAG_PAGE)); if (async) { qemu_put_buffer_async(file, buf, TARGET_PAGE_SIZE, @@ -1430,7 +1435,7 @@ static int ram_save_page(RAMState *rs, PageSearchStat= us *pss) =20 XBZRLE_cache_lock(); if (rs->xbzrle_enabled && !migration_in_postcopy()) { - pages =3D save_xbzrle_page(rs, pss->pss_channel, &p, current_addr, + pages =3D save_xbzrle_page(rs, pss, &p, current_addr, block, offset); if (!rs->last_stage) { /* Can't send this cached data async, since the cache page @@ -1442,8 +1447,7 @@ static int ram_save_page(RAMState *rs, PageSearchStat= us *pss) =20 /* XBZRLE overflow or normal page */ if (pages =3D=3D -1) { - pages =3D save_normal_page(rs, pss->pss_channel, block, offset, - p, send_async); + pages =3D save_normal_page(pss, block, offset, p, send_async); } =20 XBZRLE_cache_unlock(); @@ -1466,14 +1470,15 @@ static bool do_compress_ram_page(QEMUFile *f, z_str= eam *stream, RAMBlock *block, ram_addr_t offset, uint8_t *source_buf) { RAMState *rs =3D ram_state; + PageSearchStatus *pss =3D &rs->pss[RAM_CHANNEL_PRECOPY]; uint8_t *p =3D block->host + offset; int ret; =20 - if (save_zero_page_to_file(rs, f, block, offset)) { + if (save_zero_page_to_file(pss, block, offset)) { return true; } =20 - save_page_header(rs, f, block, offset | RAM_SAVE_FLAG_COMPRESS_PAGE); + save_page_header(pss, block, offset | RAM_SAVE_FLAG_COMPRESS_PAGE); =20 /* * copy it to a internal buffer to avoid it being modified by VM @@ -2293,7 +2298,8 @@ static bool save_page_use_compression(RAMState *rs) * has been properly handled by compression, otherwise needs other * paths to handle it */ -static bool save_compress_page(RAMState *rs, RAMBlock *block, ram_addr_t o= ffset) +static bool save_compress_page(RAMState *rs, PageSearchStatus *pss, + RAMBlock *block, ram_addr_t offset) { if (!save_page_use_compression(rs)) { return false; @@ -2309,7 +2315,7 @@ static bool save_compress_page(RAMState *rs, RAMBlock= *block, ram_addr_t offset) * We post the fist page as normal page as compression will take * much CPU resource. */ - if (block !=3D rs->last_sent_block) { + if (block !=3D pss->last_sent_block) { flush_compressed_data(rs); return false; } @@ -2340,11 +2346,11 @@ static int ram_save_target_page(RAMState *rs, PageS= earchStatus *pss) return res; } =20 - if (save_compress_page(rs, block, offset)) { + if (save_compress_page(rs, pss, block, offset)) { return 1; } =20 - res =3D save_zero_page(rs, pss->pss_channel, block, offset); + res =3D save_zero_page(pss, block, offset); if (res > 0) { /* Must let xbzrle know, otherwise a previous (now 0'd) cached * page would be stale @@ -2475,7 +2481,7 @@ static void postcopy_preempt_choose_channel(RAMState = *rs, PageSearchStatus *pss) * If channel switched, reset last_sent_block since the old sent b= lock * may not be on the same channel. */ - rs->last_sent_block =3D NULL; + pss->last_sent_block =3D NULL; =20 trace_postcopy_preempt_switch_channel(channel); } @@ -2809,8 +2815,13 @@ static void ram_save_cleanup(void *opaque) =20 static void ram_state_reset(RAMState *rs) { + int i; + + for (i =3D 0; i < RAM_CHANNEL_MAX; i++) { + rs->pss[i].last_sent_block =3D NULL; + } + rs->last_seen_block =3D NULL; - rs->last_sent_block =3D NULL; rs->last_page =3D 0; rs->last_version =3D ram_list.version; rs->xbzrle_enabled =3D false; @@ -3004,8 +3015,8 @@ void ram_postcopy_send_discard_bitmap(MigrationState = *ms) migration_bitmap_sync(rs); =20 /* Easiest way to make sure we don't resume in the middle of a host-pa= ge */ + rs->pss[RAM_CHANNEL_PRECOPY].last_sent_block =3D NULL; rs->last_seen_block =3D NULL; - rs->last_sent_block =3D NULL; rs->last_page =3D 0; =20 postcopy_each_ram_send_discard(ms); --=20 2.37.3 From nobody Sun May 19 09:23:40 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1665526700; cv=none; d=zohomail.com; s=zohoarc; b=MNImVQe5KbGq63hM+TywClozbz5DnBURFAqi65WnYiPuJ+9NXNlxVLbLv/sogpzIdWv9itVQNQwBp7KAzqW2OHZjk/ZN62TxysezB5SQ8CEH4fxuNgeVwCn6XJsmFBhhcGTFLV/uuVBP+5pALMbDB/b55hm4NdyHTg1prGxTHVU= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1665526700; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=VauCSCaXVpy+H2PmbvFmd+mT5AO5oyp8sr/leFDlpGk=; b=BIb4n5mNcyPOXAPoDOiNpiYlXYjcSDqMZU51ai+g/uY/MuuJj79HCp0L2w+up5ZlDi9fkSsxWtrz1rEKrE+aezuI6463WvVjy1lEmXq0POlmpd9HNKlD7LmnWu1qWv00O0iD7Wm5VWa+saRvr9GyctrKV+huR4emRY/+6/jV5RM= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1665526700456795.4723467260213; Tue, 11 Oct 2022 15:18:20 -0700 (PDT) Received: from localhost ([::1]:45470 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1oiNZr-0004Cj-Cx for importer@patchew.org; Tue, 11 Oct 2022 18:18:19 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:53876) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1oiNEd-0001p9-NC for qemu-devel@nongnu.org; Tue, 11 Oct 2022 17:56:24 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]:38225) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1oiNEb-0003XH-7r for qemu-devel@nongnu.org; Tue, 11 Oct 2022 17:56:23 -0400 Received: from mail-qk1-f198.google.com (mail-qk1-f198.google.com [209.85.222.198]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_128_GCM_SHA256) id us-mta-613-3yB7iMe9P6uXYBd7jyj3UQ-1; Tue, 11 Oct 2022 17:56:19 -0400 Received: by mail-qk1-f198.google.com with SMTP id bs30-20020a05620a471e00b006ed2a84071bso5024789qkb.0 for ; Tue, 11 Oct 2022 14:56:19 -0700 (PDT) Received: from x1n.redhat.com (bras-base-aurron9127w-grc-46-70-31-27-79.dsl.bell.ca. [70.31.27.79]) by smtp.gmail.com with ESMTPSA id u10-20020a05620a430a00b006ce7cd81359sm13863230qko.110.2022.10.11.14.56.17 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 11 Oct 2022 14:56:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1665525380; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=VauCSCaXVpy+H2PmbvFmd+mT5AO5oyp8sr/leFDlpGk=; b=JZ4ix8RWuLndKkgSlqG70e3siSQPAPLTYlGWI1iPe4OnmMaWLJdrsPLw8Swv59cl6ERSIf 97Pz1F85mV8usTKxzxwbNmSMfgv+ags1ixGhNXi9NUUWpS4YACa1+CZAXgFZZSrd85ekYv KXB2W2S/lNkol59D0mI+FWvmL7B+ZOU= X-MC-Unique: 3yB7iMe9P6uXYBd7jyj3UQ-1 X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=VauCSCaXVpy+H2PmbvFmd+mT5AO5oyp8sr/leFDlpGk=; b=GyyKE1FoKC0Xfz+x6D4dm1yiDiwPdJ4BHsINghufCJ7/45DmIdJ4Tu9MjikJ3aUB62 /7cESPUH7uXaQ4rQ0gHtal2FWiCrIvDACW92VqJx+gxM0qr2FOwA1CvOl++THzqKwxyp ZARHbVOQSKZ/auHl1wZVE7gWygn7qBiTWrM7txWKpxL223Y1wP5v5ahHjkre+3BZI6Vy BQoMjo/4pMr26kvyyKoz/SkVp/c0lwx4VbUhL3bj1TLF8eSPQySaKppkaQWgkf753ZqL PskOVLSGn9d21gGbGlKV1xmUQDp7FOTfNQGGkZpa58wnLxySnhcYf8kQrcGekxXlwNJX dRDg== X-Gm-Message-State: ACrzQf1UN7NzEWfwOfNpwiWnpGrnOk++fu9ME9fYMIQKEXdQDUEWqhli w7X5uaCbR5U5eakpkD02WS5b6qm341LPNY8bdYxHMsEtK3gHo6gQDppE62G+CtunlaaTDEb2SQ1 OXW+p+fjS++4dexjshzbOXF98PLrkQA7L60QbOUmZGLCw7Arvj1cF3pG/+fUTLS6X X-Received: by 2002:a05:6214:550a:b0:4b4:dd2:be99 with SMTP id mb10-20020a056214550a00b004b40dd2be99mr8692410qvb.123.1665525378671; Tue, 11 Oct 2022 14:56:18 -0700 (PDT) X-Google-Smtp-Source: AMsMyM7ntKar1ROHG3VxZkU9spjQT49BVXQODyNkvKi60mRJlqZwtSwJfaiRkoJQdzK8Svp559tNKQ== X-Received: by 2002:a05:6214:550a:b0:4b4:dd2:be99 with SMTP id mb10-20020a056214550a00b004b40dd2be99mr8692382qvb.123.1665525378356; Tue, 11 Oct 2022 14:56:18 -0700 (PDT) From: Peter Xu To: qemu-devel@nongnu.org Cc: "Dr . David Alan Gilbert" , peterx@redhat.com, "Daniel P . Berrange" , Juan Quintela , ani@anisinha.ca, Leonardo Bras Soares Passos , Manish Mishra Subject: [PATCH v2 13/15] migration: Send requested page directly in rp-return thread Date: Tue, 11 Oct 2022 17:55:57 -0400 Message-Id: <20221011215559.602584-14-peterx@redhat.com> X-Mailer: git-send-email 2.37.3 In-Reply-To: <20221011215559.602584-1-peterx@redhat.com> References: <20221011215559.602584-1-peterx@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=peterx@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -21 X-Spam_score: -2.2 X-Spam_bar: -- X-Spam_report: (-2.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URG_BIZ=0.573 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1665526702058100001 Content-Type: text/plain; charset="utf-8" With all the facilities ready, send the requested page directly in the rp-return thread rather than queuing it in the request queue, if and only if postcopy preempt is enabled. It can achieve so because it uses separate channel for sending urgent pages. The only shared data is bitmap and it's protected by the bitmap_mutex. Note that since we're moving the ownership of the urgent channel from the migration thread to rp thread it also means the rp thread is responsible for managing the qemufile, e.g. properly close it when pausing migration happens. For this, let migration_release_from_dst_file to cover shutdown of the urgent channel too, renaming it as migration_release_dst_files() to better show what it does. Reviewed-by: Dr. David Alan Gilbert Signed-off-by: Peter Xu --- migration/migration.c | 35 +++++++------ migration/ram.c | 112 ++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 131 insertions(+), 16 deletions(-) diff --git a/migration/migration.c b/migration/migration.c index 4364813d82..c0fd6588a4 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -2847,8 +2847,11 @@ static int migrate_handle_rp_resume_ack(MigrationSta= te *s, uint32_t value) return 0; } =20 -/* Release ms->rp_state.from_dst_file in a safe way */ -static void migration_release_from_dst_file(MigrationState *ms) +/* + * Release ms->rp_state.from_dst_file (and postcopy_qemufile_src if + * existed) in a safe way. + */ +static void migration_release_dst_files(MigrationState *ms) { QEMUFile *file; =20 @@ -2861,6 +2864,18 @@ static void migration_release_from_dst_file(Migratio= nState *ms) ms->rp_state.from_dst_file =3D NULL; } =20 + /* + * Do the same to postcopy fast path socket too if there is. No + * locking needed because this qemufile should only be managed by + * return path thread. + */ + if (ms->postcopy_qemufile_src) { + migration_ioc_unregister_yank_from_file(ms->postcopy_qemufile_src); + qemu_file_shutdown(ms->postcopy_qemufile_src); + qemu_fclose(ms->postcopy_qemufile_src); + ms->postcopy_qemufile_src =3D NULL; + } + qemu_fclose(file); } =20 @@ -3005,7 +3020,7 @@ out: * Maybe there is something we can do: it looks like a * network down issue, and we pause for a recovery. */ - migration_release_from_dst_file(ms); + migration_release_dst_files(ms); rp =3D NULL; if (postcopy_pause_return_path_thread(ms)) { /* @@ -3023,7 +3038,7 @@ out: } =20 trace_source_return_path_thread_end(); - migration_release_from_dst_file(ms); + migration_release_dst_files(ms); rcu_unregister_thread(); return NULL; } @@ -3546,18 +3561,6 @@ static MigThrError postcopy_pause(MigrationState *s) qemu_file_shutdown(file); qemu_fclose(file); =20 - /* - * Do the same to postcopy fast path socket too if there is. No - * locking needed because no racer as long as we do this before se= tting - * status to paused. - */ - if (s->postcopy_qemufile_src) { - migration_ioc_unregister_yank_from_file(s->postcopy_qemufile_s= rc); - qemu_file_shutdown(s->postcopy_qemufile_src); - qemu_fclose(s->postcopy_qemufile_src); - s->postcopy_qemufile_src =3D NULL; - } - migrate_set_state(&s->state, s->state, MIGRATION_STATUS_POSTCOPY_PAUSED); =20 diff --git a/migration/ram.c b/migration/ram.c index cbaa5650b8..d5a3fd610f 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -546,6 +546,8 @@ static QemuThread *decompress_threads; static QemuMutex decomp_done_lock; static QemuCond decomp_done_cond; =20 +static int ram_save_host_page_urgent(PageSearchStatus *pss); + static bool do_compress_ram_page(QEMUFile *f, z_stream *stream, RAMBlock *= block, ram_addr_t offset, uint8_t *source_buf); =20 @@ -560,6 +562,16 @@ static void pss_init(PageSearchStatus *pss, RAMBlock *= rb, ram_addr_t page) pss->complete_round =3D false; } =20 +/* + * Check whether two PSSs are actively sending the same page. Return true + * if it is, false otherwise. + */ +static bool pss_overlap(PageSearchStatus *pss1, PageSearchStatus *pss2) +{ + return pss1->host_page_sending && pss2->host_page_sending && + (pss1->host_page_start =3D=3D pss2->host_page_start); +} + static void *do_data_compress(void *opaque) { CompressParam *param =3D opaque; @@ -2260,6 +2272,57 @@ int ram_save_queue_pages(const char *rbname, ram_add= r_t start, ram_addr_t len) return -1; } =20 + /* + * When with postcopy preempt, we send back the page directly in the + * rp-return thread. + */ + if (postcopy_preempt_active()) { + ram_addr_t page_start =3D start >> TARGET_PAGE_BITS; + size_t page_size =3D qemu_ram_pagesize(ramblock); + PageSearchStatus *pss =3D &ram_state->pss[RAM_CHANNEL_POSTCOPY]; + int ret =3D 0; + + qemu_mutex_lock(&rs->bitmap_mutex); + + pss_init(pss, ramblock, page_start); + /* + * Always use the preempt channel, and make sure it's there. It's + * safe to access without lock, because when rp-thread is running + * we should be the only one who operates on the qemufile + */ + pss->pss_channel =3D migrate_get_current()->postcopy_qemufile_src; + pss->postcopy_requested =3D true; + assert(pss->pss_channel); + + /* + * It must be either one or multiple of host page size. Just + * assert; if something wrong we're mostly split brain anyway. + */ + assert(len % page_size =3D=3D 0); + while (len) { + if (ram_save_host_page_urgent(pss)) { + error_report("%s: ram_save_host_page_urgent() failed: " + "ramblock=3D%s, start_addr=3D0x"RAM_ADDR_FMT, + __func__, ramblock->idstr, start); + ret =3D -1; + break; + } + /* + * NOTE: after ram_save_host_page_urgent() succeeded, pss->page + * will automatically be moved and point to the next host page + * we're going to send, so no need to update here. + * + * Normally QEMU never sends >1 host page in requests, so + * logically we don't even need that as the loop should only + * run once, but just to be consistent. + */ + len -=3D page_size; + }; + qemu_mutex_unlock(&rs->bitmap_mutex); + + return ret; + } + struct RAMSrcPageRequest *new_entry =3D g_new0(struct RAMSrcPageRequest, 1); new_entry->rb =3D ramblock; @@ -2537,6 +2600,55 @@ static void pss_host_page_finish(PageSearchStatus *p= ss) pss->host_page_start =3D pss->host_page_end =3D 0; } =20 +/* + * Send an urgent host page specified by `pss'. Need to be called with + * bitmap_mutex held. + * + * Returns 0 if save host page succeeded, false otherwise. + */ +static int ram_save_host_page_urgent(PageSearchStatus *pss) +{ + bool page_dirty, sent =3D false; + RAMState *rs =3D ram_state; + int ret =3D 0; + + trace_postcopy_preempt_send_host_page(pss->block->idstr, pss->page); + pss_host_page_prepare(pss); + + /* + * If precopy is sending the same page, let it be done in precopy, or + * we could send the same page in two channels and none of them will + * receive the whole page. + */ + if (pss_overlap(pss, &ram_state->pss[RAM_CHANNEL_PRECOPY])) { + trace_postcopy_preempt_hit(pss->block->idstr, + pss->page << TARGET_PAGE_BITS); + return 0; + } + + do { + page_dirty =3D migration_bitmap_clear_dirty(rs, pss->block, pss->p= age); + + if (page_dirty) { + /* Be strict to return code; it must be 1, or what else? */ + if (ram_save_target_page(rs, pss) !=3D 1) { + error_report_once("%s: ram_save_target_page failed", __fun= c__); + ret =3D -1; + goto out; + } + sent =3D true; + } + pss_find_next_dirty(pss); + } while (pss_within_range(pss)); +out: + pss_host_page_finish(pss); + /* For urgent requests, flush immediately if sent */ + if (sent) { + qemu_fflush(pss->pss_channel); + } + return ret; +} + /** * ram_save_host_page: save a whole host page * --=20 2.37.3 From nobody Sun May 19 09:23:40 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1665526826; cv=none; d=zohomail.com; s=zohoarc; b=dRTXol49xET1UqAnnYEv2g8zDLMHX5sy784jBj4nWS6XIi4L9MjuGOz8A/c8/tuu9Q9ggG9KVZy1GVrtuvyVajMVpirmH6/YqzvyQTjXoqGk09utfFUpOFstZiDKqW4MYxesxkl9E53Hsjv2caFrxgbqZy3oP5PcsvCRVOH95rU= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1665526826; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=PCzM8byTZV+Lvs6hrCdlRbZDA9+G1Z+m2xzqTVxpJwY=; b=XnVhyl5o+RcjeJXYdo7Az+E/84ebAary/xnPacN0q/QxpO4Dp+wup5OTwhxUySDTg6jsFjNImPXCUQ6SVih1iTsSOrrNq4VCAPBwQ60/qSWcvdinsWaicOK0JAl7l25pRaCTnFKa48OCactG/KykRQ2LrqPP0jryjGlajd1cze8= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1665526826883941.2907939284852; Tue, 11 Oct 2022 15:20:26 -0700 (PDT) Received: from localhost ([::1]:54600 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1oiNbt-0006IK-NQ for importer@patchew.org; Tue, 11 Oct 2022 18:20:25 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:53880) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1oiNEf-0001rd-7D for qemu-devel@nongnu.org; Tue, 11 Oct 2022 17:56:25 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]:27146) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1oiNEc-0003Xq-L3 for qemu-devel@nongnu.org; Tue, 11 Oct 2022 17:56:24 -0400 Received: from mail-qt1-f199.google.com (mail-qt1-f199.google.com [209.85.160.199]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_128_GCM_SHA256) id us-mta-189-ahgsWSCiPeKU9saEVZdBIA-1; Tue, 11 Oct 2022 17:56:21 -0400 Received: by mail-qt1-f199.google.com with SMTP id ew15-20020a05622a514f00b0035cb76e35e5so9926954qtb.7 for ; Tue, 11 Oct 2022 14:56:20 -0700 (PDT) Received: from x1n.redhat.com (bras-base-aurron9127w-grc-46-70-31-27-79.dsl.bell.ca. [70.31.27.79]) by smtp.gmail.com with ESMTPSA id u10-20020a05620a430a00b006ce7cd81359sm13863230qko.110.2022.10.11.14.56.18 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 11 Oct 2022 14:56:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1665525382; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=PCzM8byTZV+Lvs6hrCdlRbZDA9+G1Z+m2xzqTVxpJwY=; b=KxnEreEm/Q6NNumwQuoAUWwjEKPyJSMjPyTize7a6GO+4gZC7SA38UoZPzPUycPQNjYr3H HOHHvdB7NeBX07ARx31npoR3K+g73XZGsDI5pjUHwI8UYN6yI3XWGj4vN6q0PwMzDKQMz0 WbXc4w9qlTeYCwLXA0A/9123mprnCvo= X-MC-Unique: ahgsWSCiPeKU9saEVZdBIA-1 X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=PCzM8byTZV+Lvs6hrCdlRbZDA9+G1Z+m2xzqTVxpJwY=; b=kEu97cPCusj+K1JYEjqiIPMTngrXA9XZQfcg4DNA6Q/hkyIyoeLs52MwmBZh9Gs8NT TQNswGss+X2++qKBbpVilSxJ0YnymP2YeVXQEf+9a+w+IAXYECM6zxxBP0O9GqGcKXOb xkYdQQluOVBEoot5PjQzxHPwbDeb12kT43MMY+8W4qc7c8nWr6cYehk70Ft3TX76CPos f7S9pzmPjUenL9CwGx2EYJVEzDg77YFmWZAvvcKvc1Q7says8ovtIUP5hBvRh9IVh65B 3bzQBWdi5J5Ih82GTz5fmFHj3aKd6FDOb5b72cCa3hyNqaRc+VyxvGhEk93dwric2Mrq BJWg== X-Gm-Message-State: ACrzQf1dhoQsaeqzMypB6qyf9BJA7caLuzSZUCf56q8uWwanr/aVrrx7 DvqnroWc+Ux6ZxmHv3dEdmu3w2voYyJvk42ys4iWWKJNkUNN6hCBUN8p3KnI8snA+oZ/uC5wCP+ Eu+GlfQkJMKDt19ygQ/TQjO2bBD37Y84CcqoTdOcQE6kn7T595ODSH2RZPvzCRuNS X-Received: by 2002:ae9:e401:0:b0:6e5:291d:4073 with SMTP id q1-20020ae9e401000000b006e5291d4073mr18172922qkc.635.1665525380083; Tue, 11 Oct 2022 14:56:20 -0700 (PDT) X-Google-Smtp-Source: AMsMyM4S33ILJlmkZuzYuDXGV1E6myG9wWsENcn+qZEz6k1MkSLnQYfG84qWRLkXGztVnNpvmebmfQ== X-Received: by 2002:ae9:e401:0:b0:6e5:291d:4073 with SMTP id q1-20020ae9e401000000b006e5291d4073mr18172901qkc.635.1665525379554; Tue, 11 Oct 2022 14:56:19 -0700 (PDT) From: Peter Xu To: qemu-devel@nongnu.org Cc: "Dr . David Alan Gilbert" , peterx@redhat.com, "Daniel P . Berrange" , Juan Quintela , ani@anisinha.ca, Leonardo Bras Soares Passos , Manish Mishra Subject: [PATCH v2 14/15] migration: Remove old preempt code around state maintainance Date: Tue, 11 Oct 2022 17:55:58 -0400 Message-Id: <20221011215559.602584-15-peterx@redhat.com> X-Mailer: git-send-email 2.37.3 In-Reply-To: <20221011215559.602584-1-peterx@redhat.com> References: <20221011215559.602584-1-peterx@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=peterx@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -21 X-Spam_score: -2.2 X-Spam_bar: -- X-Spam_report: (-2.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URG_BIZ=0.573 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1665526828854100001 Content-Type: text/plain; charset="utf-8" With the new code to send pages in rp-return thread, there's little help to keep lots of the old code on maintaining the preempt state in migration thread, because the new way should always be faster.. Then if we'll always send pages in the rp-return thread anyway, we don't need those logic to maintain preempt state anymore because now we serialize things using the mutex directly instead of using those fields. It's very unfortunate to have those code for a short period, but that's still one intermediate step that we noticed the next bottleneck on the migration thread. Now what we can do best is to drop unnecessary code as long as the new code is stable to reduce the burden. It's actually a good thing because the new "sending page in rp-return thread" model is (IMHO) even cleaner and with better performance. Remove the old code that was responsible for maintaining preempt states, at the meantime also remove x-postcopy-preempt-break-huge parameter because with concurrent sender threads we don't really need to break-huge anymore. Reviewed-by: Dr. David Alan Gilbert Signed-off-by: Peter Xu Reviewed-by: Juan Quintela --- migration/migration.c | 2 - migration/migration.h | 7 - migration/ram.c | 291 +----------------------------------------- 3 files changed, 3 insertions(+), 297 deletions(-) diff --git a/migration/migration.c b/migration/migration.c index c0fd6588a4..59c6aa3960 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -4401,8 +4401,6 @@ static Property migration_properties[] =3D { DEFINE_PROP_SIZE("announce-step", MigrationState, parameters.announce_step, DEFAULT_MIGRATE_ANNOUNCE_STEP), - DEFINE_PROP_BOOL("x-postcopy-preempt-break-huge", MigrationState, - postcopy_preempt_break_huge, true), DEFINE_PROP_STRING("tls-creds", MigrationState, parameters.tls_creds), DEFINE_PROP_STRING("tls-hostname", MigrationState, parameters.tls_host= name), DEFINE_PROP_STRING("tls-authz", MigrationState, parameters.tls_authz), diff --git a/migration/migration.h b/migration/migration.h index cdad8aceaa..ae4ffd3454 100644 --- a/migration/migration.h +++ b/migration/migration.h @@ -340,13 +340,6 @@ struct MigrationState { bool send_configuration; /* Whether we send section footer during migration */ bool send_section_footer; - /* - * Whether we allow break sending huge pages when postcopy preempt is - * enabled. When disabled, we won't interrupt precopy within sending a - * host huge page, which is the old behavior of vanilla postcopy. - * NOTE: this parameter is ignored if postcopy preempt is not enabled. - */ - bool postcopy_preempt_break_huge; =20 /* Needed by postcopy-pause state */ QemuSemaphore postcopy_pause_sem; diff --git a/migration/ram.c b/migration/ram.c index d5a3fd610f..db3bf51dad 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -97,28 +97,6 @@ struct PageSearchStatus { unsigned long page; /* Set once we wrap around */ bool complete_round; - /* - * [POSTCOPY-ONLY] Whether current page is explicitly requested by - * postcopy. When set, the request is "urgent" because the dest QEMU - * threads are waiting for us. - */ - bool postcopy_requested; - /* - * [POSTCOPY-ONLY] The target channel to use to send current page. - * - * Note: This may _not_ match with the value in postcopy_requested - * above. Let's imagine the case where the postcopy request is exactly - * the page that we're sending in progress during precopy. In this case - * we'll have postcopy_requested set to true but the target channel - * will be the precopy channel (so that we don't split brain on that - * specific page since the precopy channel already contains partial of - * that page data). - * - * Besides that specific use case, postcopy_target_channel should - * always be equal to postcopy_requested, because by default we send - * postcopy pages via postcopy preempt channel. - */ - bool postcopy_target_channel; /* Whether we're sending a host page */ bool host_page_sending; /* The start/end of current host page. Invalid if host_page_sending= =3D=3Dfalse */ @@ -343,20 +321,6 @@ struct RAMSrcPageRequest { QSIMPLEQ_ENTRY(RAMSrcPageRequest) next_req; }; =20 -typedef struct { - /* - * Cached ramblock/offset values if preempted. They're only meaningfu= l if - * preempted=3D=3Dtrue below. - */ - RAMBlock *ram_block; - unsigned long ram_page; - /* - * Whether a postcopy preemption just happened. Will be reset after - * precopy recovered to background migration. - */ - bool preempted; -} PostcopyPreemptState; - /* State of RAM for migration */ struct RAMState { /* QEMUFile used for this migration */ @@ -419,14 +383,6 @@ struct RAMState { /* Queue of outstanding page requests from the destination */ QemuMutex src_page_req_mutex; QSIMPLEQ_HEAD(, RAMSrcPageRequest) src_page_requests; - - /* Postcopy preemption informations */ - PostcopyPreemptState postcopy_preempt_state; - /* - * Current channel we're using on src VM. Only valid if postcopy-pree= mpt - * is enabled. - */ - unsigned int postcopy_channel; }; typedef struct RAMState RAMState; =20 @@ -434,11 +390,6 @@ static RAMState *ram_state; =20 static NotifierWithReturnList precopy_notifier_list; =20 -static void postcopy_preempt_reset(RAMState *rs) -{ - memset(&rs->postcopy_preempt_state, 0, sizeof(PostcopyPreemptState)); -} - /* Whether postcopy has queued requests? */ static bool postcopy_has_request(RAMState *rs) { @@ -551,9 +502,6 @@ static int ram_save_host_page_urgent(PageSearchStatus *= pss); static bool do_compress_ram_page(QEMUFile *f, z_stream *stream, RAMBlock *= block, ram_addr_t offset, uint8_t *source_buf); =20 -static void postcopy_preempt_restore(RAMState *rs, PageSearchStatus *pss, - bool postcopy_requested); - /* NOTE: page is the PFN not real ram_addr_t. */ static void pss_init(PageSearchStatus *pss, RAMBlock *rb, ram_addr_t page) { @@ -1612,13 +1560,6 @@ retry: */ static bool find_dirty_block(RAMState *rs, PageSearchStatus *pss, bool *ag= ain) { - /* - * This is not a postcopy requested page, mark it "not urgent", and use - * precopy channel to send it. - */ - pss->postcopy_requested =3D false; - pss->postcopy_target_channel =3D RAM_CHANNEL_PRECOPY; - /* Update pss->page for the next dirty bit in ramblock */ pss_find_next_dirty(pss); =20 @@ -2069,55 +2010,6 @@ void ram_write_tracking_stop(void) } #endif /* defined(__linux__) */ =20 -/* - * Check whether two addr/offset of the ramblock falls onto the same host = huge - * page. Returns true if so, false otherwise. - */ -static bool offset_on_same_huge_page(RAMBlock *rb, uint64_t addr1, - uint64_t addr2) -{ - size_t page_size =3D qemu_ram_pagesize(rb); - - addr1 =3D ROUND_DOWN(addr1, page_size); - addr2 =3D ROUND_DOWN(addr2, page_size); - - return addr1 =3D=3D addr2; -} - -/* - * Whether a previous preempted precopy huge page contains current request= ed - * page? Returns true if so, false otherwise. - * - * This should really happen very rarely, because it means when we were se= nding - * during background migration for postcopy we're sending exactly the page= that - * some vcpu got faulted on on dest node. When it happens, we probably do= n't - * need to do much but drop the request, because we know right after we re= store - * the precopy stream it'll be serviced. It'll slightly affect the order = of - * postcopy requests to be serviced (e.g. it'll be the same as we move cur= rent - * request to the end of the queue) but it shouldn't be a big deal. The m= ost - * imporant thing is we can _never_ try to send a partial-sent huge page o= n the - * POSTCOPY channel again, otherwise that huge page will got "split brain"= on - * two channels (PRECOPY, POSTCOPY). - */ -static bool postcopy_preempted_contains(RAMState *rs, RAMBlock *block, - ram_addr_t offset) -{ - PostcopyPreemptState *state =3D &rs->postcopy_preempt_state; - - /* No preemption at all? */ - if (!state->preempted) { - return false; - } - - /* Not even the same ramblock? */ - if (state->ram_block !=3D block) { - return false; - } - - return offset_on_same_huge_page(block, offset, - state->ram_page << TARGET_PAGE_BITS); -} - /** * get_queued_page: unqueue a page from the postcopy requests * @@ -2157,20 +2049,7 @@ static bool get_queued_page(RAMState *rs, PageSearch= Status *pss) =20 } while (block && !dirty); =20 - if (block) { - /* See comment above postcopy_preempted_contains() */ - if (postcopy_preempted_contains(rs, block, offset)) { - trace_postcopy_preempt_hit(block->idstr, offset); - /* - * If what we preempted previously was exactly what we're - * requesting right now, restore the preempted precopy - * immediately, boosting its priority as it's requested by - * postcopy. - */ - postcopy_preempt_restore(rs, pss, true); - return true; - } - } else { + if (!block) { /* * Poll write faults too if background snapshot is enabled; that's * when we have vcpus got blocked by the write protected pages. @@ -2192,9 +2071,6 @@ static bool get_queued_page(RAMState *rs, PageSearchS= tatus *pss) * really rare. */ pss->complete_round =3D false; - /* Mark it an urgent request, meanwhile using POSTCOPY channel */ - pss->postcopy_requested =3D true; - pss->postcopy_target_channel =3D RAM_CHANNEL_POSTCOPY; } =20 return !!block; @@ -2291,7 +2167,6 @@ int ram_save_queue_pages(const char *rbname, ram_addr= _t start, ram_addr_t len) * we should be the only one who operates on the qemufile */ pss->pss_channel =3D migrate_get_current()->postcopy_qemufile_src; - pss->postcopy_requested =3D true; assert(pss->pss_channel); =20 /* @@ -2439,129 +2314,6 @@ static int ram_save_target_page(RAMState *rs, PageS= earchStatus *pss) return ram_save_page(rs, pss); } =20 -static bool postcopy_needs_preempt(RAMState *rs, PageSearchStatus *pss) -{ - MigrationState *ms =3D migrate_get_current(); - - /* Not enabled eager preempt? Then never do that. */ - if (!migrate_postcopy_preempt()) { - return false; - } - - /* If the user explicitly disabled breaking of huge page, skip */ - if (!ms->postcopy_preempt_break_huge) { - return false; - } - - /* If the ramblock we're sending is a small page? Never bother. */ - if (qemu_ram_pagesize(pss->block) =3D=3D TARGET_PAGE_SIZE) { - return false; - } - - /* Not in postcopy at all? */ - if (!migration_in_postcopy()) { - return false; - } - - /* - * If we're already handling a postcopy request, don't preempt as this= page - * has got the same high priority. - */ - if (pss->postcopy_requested) { - return false; - } - - /* If there's postcopy requests, then check it up! */ - return postcopy_has_request(rs); -} - -/* Returns true if we preempted precopy, false otherwise */ -static void postcopy_do_preempt(RAMState *rs, PageSearchStatus *pss) -{ - PostcopyPreemptState *p_state =3D &rs->postcopy_preempt_state; - - trace_postcopy_preempt_triggered(pss->block->idstr, pss->page); - - /* - * Time to preempt precopy. Cache current PSS into preempt state, so t= hat - * after handling the postcopy pages we can recover to it. We need to= do - * so because the dest VM will have partial of the precopy huge page k= ept - * over in its tmp huge page caches; better move on with it when we ca= n. - */ - p_state->ram_block =3D pss->block; - p_state->ram_page =3D pss->page; - p_state->preempted =3D true; -} - -/* Whether we're preempted by a postcopy request during sending a huge pag= e */ -static bool postcopy_preempt_triggered(RAMState *rs) -{ - return rs->postcopy_preempt_state.preempted; -} - -static void postcopy_preempt_restore(RAMState *rs, PageSearchStatus *pss, - bool postcopy_requested) -{ - PostcopyPreemptState *state =3D &rs->postcopy_preempt_state; - - assert(state->preempted); - - pss->block =3D state->ram_block; - pss->page =3D state->ram_page; - - /* Whether this is a postcopy request? */ - pss->postcopy_requested =3D postcopy_requested; - /* - * When restoring a preempted page, the old data resides in PRECOPY - * slow channel, even if postcopy_requested is set. So always use - * PRECOPY channel here. - */ - pss->postcopy_target_channel =3D RAM_CHANNEL_PRECOPY; - - trace_postcopy_preempt_restored(pss->block->idstr, pss->page); - - /* Reset preempt state, most importantly, set preempted=3D=3Dfalse */ - postcopy_preempt_reset(rs); -} - -static void postcopy_preempt_choose_channel(RAMState *rs, PageSearchStatus= *pss) -{ - MigrationState *s =3D migrate_get_current(); - unsigned int channel =3D pss->postcopy_target_channel; - QEMUFile *next; - - if (channel !=3D rs->postcopy_channel) { - if (channel =3D=3D RAM_CHANNEL_PRECOPY) { - next =3D s->to_dst_file; - } else { - next =3D s->postcopy_qemufile_src; - } - /* Update and cache the current channel */ - rs->f =3D next; - rs->postcopy_channel =3D channel; - - /* - * If channel switched, reset last_sent_block since the old sent b= lock - * may not be on the same channel. - */ - pss->last_sent_block =3D NULL; - - trace_postcopy_preempt_switch_channel(channel); - } - - trace_postcopy_preempt_send_host_page(pss->block->idstr, pss->page); -} - -/* We need to make sure rs->f always points to the default channel elsewhe= re */ -static void postcopy_preempt_reset_channel(RAMState *rs) -{ - if (postcopy_preempt_active()) { - rs->postcopy_channel =3D RAM_CHANNEL_PRECOPY; - rs->f =3D migrate_get_current()->to_dst_file; - trace_postcopy_preempt_reset_channel(); - } -} - /* Should be called before sending a host page */ static void pss_host_page_prepare(PageSearchStatus *pss) { @@ -2683,11 +2435,6 @@ static int ram_save_host_page(RAMState *rs, PageSear= chStatus *pss) pss_host_page_prepare(pss); =20 do { - if (postcopy_needs_preempt(rs, pss)) { - postcopy_do_preempt(rs, pss); - break; - } - page_dirty =3D migration_bitmap_clear_dirty(rs, pss->block, pss->p= age); =20 /* Check the pages is dirty and if it is send it */ @@ -2728,19 +2475,6 @@ static int ram_save_host_page(RAMState *rs, PageSear= chStatus *pss) =20 pss_host_page_finish(pss); =20 - /* - * When with postcopy preempt mode, flush the data as soon as possible= for - * postcopy requests, because we've already sent a whole huge page, so= the - * dst node should already have enough resource to atomically filling = in - * the current missing page. - * - * More importantly, when using separate postcopy channel, we must do - * explicit flush or it won't flush until the buffer is full. - */ - if (migrate_postcopy_preempt() && pss->postcopy_requested) { - qemu_fflush(pss->pss_channel); - } - res =3D ram_save_release_protection(rs, pss, start_page); return (res < 0 ? res : pages); } @@ -2788,24 +2522,11 @@ static int ram_find_and_save_block(RAMState *rs) found =3D get_queued_page(rs, pss); =20 if (!found) { - /* - * Recover previous precopy ramblock/offset if postcopy has - * preempted precopy. Otherwise find the next dirty bit. - */ - if (postcopy_preempt_triggered(rs)) { - postcopy_preempt_restore(rs, pss, false); - found =3D true; - } else { - /* priority queue empty, so just search for something dirt= y */ - found =3D find_dirty_block(rs, pss, &again); - } + /* priority queue empty, so just search for something dirty */ + found =3D find_dirty_block(rs, pss, &again); } =20 if (found) { - /* Update rs->f with correct channel */ - if (postcopy_preempt_active()) { - postcopy_preempt_choose_channel(rs, pss); - } /* Cache rs->f in pss_channel (TODO: remove rs->f) */ pss->pss_channel =3D rs->f; pages =3D ram_save_host_page(rs, pss); @@ -2937,8 +2658,6 @@ static void ram_state_reset(RAMState *rs) rs->last_page =3D 0; rs->last_version =3D ram_list.version; rs->xbzrle_enabled =3D false; - postcopy_preempt_reset(rs); - rs->postcopy_channel =3D RAM_CHANNEL_PRECOPY; } =20 #define MAX_WAIT 50 /* ms, half buffered_file limit */ @@ -3582,8 +3301,6 @@ static int ram_save_iterate(QEMUFile *f, void *opaque) } qemu_mutex_unlock(&rs->bitmap_mutex); =20 - postcopy_preempt_reset_channel(rs); - /* * Must occur before EOS (or any QEMUFile operation) * because of RDMA protocol. @@ -3663,8 +3380,6 @@ static int ram_save_complete(QEMUFile *f, void *opaqu= e) return ret; } =20 - postcopy_preempt_reset_channel(rs); - ret =3D multifd_send_sync_main(rs->f); if (ret < 0) { return ret; --=20 2.37.3 From nobody Sun May 19 09:23:40 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1665526303; cv=none; d=zohomail.com; s=zohoarc; b=PSzAvhcsjHYhRm950wox2cS9tMjLpSMqmKVZTIuqqm1SBCxqq4IW+ouvM3VLKC/h8Ju4TMYPeTJgayEueR81Wz0HwpEmN1IO7woINbNuyrJAav0wy1gTTh08ZH0Lek9ksZQJGYHIfp3+0CwQIt9LURZKeWbqQsw1fKJfqQocYUk= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1665526303; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=jD7tsKbZ/y7WJ9tIIuvB2nn8zEDHeThpW1yJ/5VkuHs=; b=BM+Ng9hgQYty3SvUkmGBTgrqvNkLLpaG4N9g6QIum06lv2U9U/O4gqawkPOpfTxBM8btQYRTBzMkFVHUcSGVFaUTmKGXYud4jG9Xlv3KGMfmYU3DtdK5xt/l5lA1+wOeTFrVHh8bm3K737qwH+KrgvYsD0lUAmIp0HVW/zvfBSA= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 166552630335713.057876567059338; Tue, 11 Oct 2022 15:11:43 -0700 (PDT) Received: from localhost ([::1]:35718 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1oiNTS-0000vO-80 for importer@patchew.org; Tue, 11 Oct 2022 18:11:42 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:53878) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1oiNEe-0001rJ-SX for qemu-devel@nongnu.org; Tue, 11 Oct 2022 17:56:25 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]:55156) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1oiNEd-0003YM-4W for qemu-devel@nongnu.org; Tue, 11 Oct 2022 17:56:24 -0400 Received: from mail-qt1-f197.google.com (mail-qt1-f197.google.com [209.85.160.197]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_128_GCM_SHA256) id us-mta-191-QDjEt3ZINUmfJ_es7l2hqw-1; Tue, 11 Oct 2022 17:56:21 -0400 Received: by mail-qt1-f197.google.com with SMTP id bz12-20020a05622a1e8c00b0039ae6e887ffso3833489qtb.8 for ; Tue, 11 Oct 2022 14:56:21 -0700 (PDT) Received: from x1n.redhat.com (bras-base-aurron9127w-grc-46-70-31-27-79.dsl.bell.ca. [70.31.27.79]) by smtp.gmail.com with ESMTPSA id u10-20020a05620a430a00b006ce7cd81359sm13863230qko.110.2022.10.11.14.56.19 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 11 Oct 2022 14:56:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1665525382; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=jD7tsKbZ/y7WJ9tIIuvB2nn8zEDHeThpW1yJ/5VkuHs=; b=JRBZTYUfLez5ja6k10+4C6loC885FB+nbaf8T7mPusUcZ11b7EtI7DRArp2XWI/dkxT0vt 7fp8QBhTVlaCurMO10gCcN7MnqRr+BWjlK4fU05rNVEGminBbJVPkjT6HiX+8OBaXiXmYn 4NR4bdktF87g+5rgK9NrkxpByKvj+0s= X-MC-Unique: QDjEt3ZINUmfJ_es7l2hqw-1 X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=jD7tsKbZ/y7WJ9tIIuvB2nn8zEDHeThpW1yJ/5VkuHs=; b=xMCb8fhFp+Fx/HJz489E1hlZXJvLeHYitxQ5qn4rrwRSwpK4BZXxAuD4tKcJvNelyj Gp9tipV3M4Sbc8WIjYv4gKmcug3AgWOSRet5a+NNYTxHHZJJu8gPyuJ8xRTYMERifybe 25Xh4efr2rkmvNCOw1+6ioxEWW8C/8Uv2Jm0ofyqVuKzSQIfX8VQtU0m7YubquL04Ovb +1p+DgoO5XPf89iZBENUAw4I0aA8ZErCqgHq5nlQ+gVVMmHCjF9cttEsEJ5ya+Mpbha0 CgZMjsZ5DtCnfNJmLFr6FUZAh4lg/AqrAMCoPkklSzCrqR6QvqlXBlKf828VK/Uc9GJK Lexw== X-Gm-Message-State: ACrzQf1nt+PJsL0qxqqjs3nldmkE0UfMF+zyrOOQsjQSJHvtX8jdVRVU Sjx+sXHsQy68L8XFnoCfuCh3CSdVC4rw3ShJlsMSP3HZfkdkrd2OfxreEt1bCjqZMyyq74VFBPA 2jc+ruwxW2lWpPkwc8aEUGfMcNDGA3gX74frFr07lMQ7fHqtpKRqm/bjUNJOrQ9c7 X-Received: by 2002:a05:620a:d58:b0:6df:ba25:4f5e with SMTP id o24-20020a05620a0d5800b006dfba254f5emr17973290qkl.320.1665525380878; Tue, 11 Oct 2022 14:56:20 -0700 (PDT) X-Google-Smtp-Source: AMsMyM7t0dtiPlqrHrYcxZX10L2zlFeHiCzt9ZMvXhaXG4xUM5m++creuOF4Tn1u2IOXSgVngmbZpA== X-Received: by 2002:a05:620a:d58:b0:6df:ba25:4f5e with SMTP id o24-20020a05620a0d5800b006dfba254f5emr17973272qkl.320.1665525380590; Tue, 11 Oct 2022 14:56:20 -0700 (PDT) From: Peter Xu To: qemu-devel@nongnu.org Cc: "Dr . David Alan Gilbert" , peterx@redhat.com, "Daniel P . Berrange" , Juan Quintela , ani@anisinha.ca, Leonardo Bras Soares Passos , Manish Mishra Subject: [PATCH v2 15/15] migration: Drop rs->f Date: Tue, 11 Oct 2022 17:55:59 -0400 Message-Id: <20221011215559.602584-16-peterx@redhat.com> X-Mailer: git-send-email 2.37.3 In-Reply-To: <20221011215559.602584-1-peterx@redhat.com> References: <20221011215559.602584-1-peterx@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=peterx@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1665526304330100003 Content-Type: text/plain; charset="utf-8" Now with rs->pss we can already cache channels in pss->pss_channels. That pss_channel contains more infromation than rs->f because it's per-channel. So rs->f could be replaced by rss->pss[RAM_CHANNEL_PRECOPY].pss_channel, while rs->f itself is a bit vague now. Note that vanilla postcopy still send pages via pss[RAM_CHANNEL_PRECOPY], that's slightly confusing but it reflects the reality. Then, after the replacement we can safely drop rs->f. Reviewed-by: Dr. David Alan Gilbert Signed-off-by: Peter Xu Reviewed-by: Juan Quintela --- migration/ram.c | 12 ++++-------- 1 file changed, 4 insertions(+), 8 deletions(-) diff --git a/migration/ram.c b/migration/ram.c index db3bf51dad..538667b974 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -323,8 +323,6 @@ struct RAMSrcPageRequest { =20 /* State of RAM for migration */ struct RAMState { - /* QEMUFile used for this migration */ - QEMUFile *f; /* * PageSearchStatus structures for the channels when send pages. * Protected by the bitmap_mutex. @@ -2527,8 +2525,6 @@ static int ram_find_and_save_block(RAMState *rs) } =20 if (found) { - /* Cache rs->f in pss_channel (TODO: remove rs->f) */ - pss->pss_channel =3D rs->f; pages =3D ram_save_host_page(rs, pss); } } while (!pages && again); @@ -3084,7 +3080,7 @@ static void ram_state_resume_prepare(RAMState *rs, QE= MUFile *out) ram_state_reset(rs); =20 /* Update RAMState cache of output QEMUFile */ - rs->f =3D out; + rs->pss[RAM_CHANNEL_PRECOPY].pss_channel =3D out; =20 trace_ram_state_resume_prepare(pages); } @@ -3175,7 +3171,7 @@ static int ram_save_setup(QEMUFile *f, void *opaque) return -1; } } - (*rsp)->f =3D f; + (*rsp)->pss[RAM_CHANNEL_PRECOPY].pss_channel =3D f; =20 WITH_RCU_READ_LOCK_GUARD() { qemu_put_be64(f, ram_bytes_total_common(true) | RAM_SAVE_FLAG_MEM_= SIZE); @@ -3310,7 +3306,7 @@ static int ram_save_iterate(QEMUFile *f, void *opaque) out: if (ret >=3D 0 && migration_is_setup_or_active(migrate_get_current()->state)) { - ret =3D multifd_send_sync_main(rs->f); + ret =3D multifd_send_sync_main(rs->pss[RAM_CHANNEL_PRECOPY].pss_ch= annel); if (ret < 0) { return ret; } @@ -3380,7 +3376,7 @@ static int ram_save_complete(QEMUFile *f, void *opaqu= e) return ret; } =20 - ret =3D multifd_send_sync_main(rs->f); + ret =3D multifd_send_sync_main(rs->pss[RAM_CHANNEL_PRECOPY].pss_channe= l); if (ret < 0) { return ret; } --=20 2.37.3