From nobody Fri May 17 02:03:38 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1682709269; cv=none; d=zohomail.com; s=zohoarc; b=c8KYmgph5fbqnGYgxEslDecxvDGxfKp8XvPK9nY/VbE911+/p11SE/p7CStzo0B7E7YGP0ZnqPTc+fT+5IXwY8BU8xtl6EWsQnvTcge1OcdP4OaEuVMMUbymisW6eMJITBPLgbWj8+Nw/sYZN6QoJOPBZEeQuaZGeT4PLs6GVOs= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1682709269; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=x//nluauCeOMTwlVk/MtZbhxlX1fdBq+vgsF2NUk3W0=; b=j37tR0a8UU+gto3Ty7yYDjtyQOb97RTFjyj28SNTPceBMFsxfxvDfp5eNxhfQBvW8ZqlR4rC8JrpdIZsqVHEB0XogkXrLrwlxVijRAN2fhcukctgkE7YJtOR7CAVJT1MHbD38bmkZcje63yPiHkAJJp8flzolRMbT5N74bNp1Tw= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1682709269503824.6573232152342; Fri, 28 Apr 2023 12:14:29 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1psTW8-0004ia-BI; Fri, 28 Apr 2023 15:12:28 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1psTVt-0004VU-Hs for qemu-devel@nongnu.org; Fri, 28 Apr 2023 15:12:15 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1psTVs-0002u3-5m for qemu-devel@nongnu.org; Fri, 28 Apr 2023 15:12:13 -0400 Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-453-YFrvLPuBPROqfaVPewq2nQ-1; Fri, 28 Apr 2023 15:12:07 -0400 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.rdu2.redhat.com [10.11.54.6]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 4D67C185A78B; Fri, 28 Apr 2023 19:12:07 +0000 (UTC) Received: from secure.mitica (unknown [10.39.193.238]) by smtp.corp.redhat.com (Postfix) with ESMTP id BBC622166B4F; Fri, 28 Apr 2023 19:12:05 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1682709131; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=x//nluauCeOMTwlVk/MtZbhxlX1fdBq+vgsF2NUk3W0=; b=fr+YFHgTtXgXEjkvuWpiu6LCrmm19nd1xKrj77uy36XugxcqGIfxySdKhIoq978iF08FTl sdjSKXiJ8qr935peHqm7WW5KfysPo310sEE1Zxu6A3Tgbq+pem55X/BwtMb1yn5k+5/Zwb jadjUvrCqiZL8mjIommjs19SILdIsz0= X-MC-Unique: YFrvLPuBPROqfaVPewq2nQ-1 From: Juan Quintela To: qemu-devel@nongnu.org Cc: Leonardo Bras , Juan Quintela , Thomas Huth , Laurent Vivier , Paolo Bonzini , Peter Xu , Lukas Straub Subject: [PULL 01/21] multifd: We already account for this packet on the multifd thread Date: Fri, 28 Apr 2023 21:11:43 +0200 Message-Id: <20230428191203.39520-2-quintela@redhat.com> In-Reply-To: <20230428191203.39520-1-quintela@redhat.com> References: <20230428191203.39520-1-quintela@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.1 on 10.11.54.6 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=quintela@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -22 X-Spam_score: -2.3 X-Spam_bar: -- X-Spam_report: (-2.3 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.171, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1682709271088100007 Content-Type: text/plain; charset="utf-8" Signed-off-by: Juan Quintela Reviewed-by: Lukas Straub --- migration/multifd.c | 3 --- 1 file changed, 3 deletions(-) diff --git a/migration/multifd.c b/migration/multifd.c index 6a59c03dd2..6053012ad9 100644 --- a/migration/multifd.c +++ b/migration/multifd.c @@ -626,10 +626,7 @@ int multifd_send_sync_main(QEMUFile *f) p->packet_num =3D multifd_send_state->packet_num++; p->flags |=3D MULTIFD_FLAG_SYNC; p->pending_job++; - qemu_file_acct_rate_limit(f, p->packet_len); qemu_mutex_unlock(&p->mutex); - stat64_add(&ram_counters.transferred, p->packet_len); - stat64_add(&ram_counters.multifd_bytes, p->packet_len); qemu_sem_post(&p->sem); } for (i =3D 0; i < migrate_multifd_channels(); i++) { --=20 2.40.0 From nobody Fri May 17 02:03:38 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1682709171; cv=none; d=zohomail.com; s=zohoarc; b=aLKuPz0CXRgA5XrXIsw/HGJb0NGek+BXM2io5bTR9rDaJIlzAxU3wgwTzfxdDOnAPq9Gnbcv2ndxWRhxcOu0ig+CDt7b1nqVyNA1aJX+Z8CNkKRKXl5/oIwJ8eQ55SIkL9J9NQ5HDugOPoZc4NHCv2PSIqLKzlEOJ1ACYtAAWB4= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1682709171; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=vSCB6KT7M6hoNQV4Q2mURuJbWzf3pyiKoUhw7b9R8Xc=; b=TmJtIouhDFrv3RmzrsnX3dWQFRHoWPBEAJ5+MaTq7si8SDWpelZ4MU9tGjT0qQAkblVE7CzlyQo+cZWGC8k2NnpkFuFFhxrHcn/8eQllWwAiTMU4oiHfzgePR3DytooFZ+8qKtPKg7g9GRh+HEW+rCVtawj/bX7A+Q79+ph2aQ8= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1682709171424604.3318629640003; Fri, 28 Apr 2023 12:12:51 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1psTW7-0004f0-9f; Fri, 28 Apr 2023 15:12:27 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1psTVt-0004VV-Ti for qemu-devel@nongnu.org; Fri, 28 Apr 2023 15:12:15 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1psTVs-0002u4-6B for qemu-devel@nongnu.org; Fri, 28 Apr 2023 15:12:13 -0400 Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-13-yapo_36qPxe5g4LYr2T02g-1; Fri, 28 Apr 2023 15:12:09 -0400 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.rdu2.redhat.com [10.11.54.6]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 2224F1C06916; Fri, 28 Apr 2023 19:12:09 +0000 (UTC) Received: from secure.mitica (unknown [10.39.193.238]) by smtp.corp.redhat.com (Postfix) with ESMTP id 8F84B2166B4F; Fri, 28 Apr 2023 19:12:07 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1682709131; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=vSCB6KT7M6hoNQV4Q2mURuJbWzf3pyiKoUhw7b9R8Xc=; b=a2acTRzT17+4RfY9WsPzQ4qO650d0WQBro3njiqAvK8bnSymGK5LA62/KP1K7wXD6ztWVL E6cCoYIwW3KUJUOacexI7iOEX7Lp0ysSZhjzuQJ6Ajveyr7vWCLMfGgR3E5OBzB+UJCvi7 4z6EFO7lAk36qWdxULLf3KGfXrCZooY= X-MC-Unique: yapo_36qPxe5g4LYr2T02g-1 From: Juan Quintela To: qemu-devel@nongnu.org Cc: Leonardo Bras , Juan Quintela , Thomas Huth , Laurent Vivier , Paolo Bonzini , Peter Xu , Lukas Straub Subject: [PULL 02/21] migration: Move ram_stats to its own file migration-stats.[ch] Date: Fri, 28 Apr 2023 21:11:44 +0200 Message-Id: <20230428191203.39520-3-quintela@redhat.com> In-Reply-To: <20230428191203.39520-1-quintela@redhat.com> References: <20230428191203.39520-1-quintela@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.1 on 10.11.54.6 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=quintela@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -22 X-Spam_score: -2.3 X-Spam_bar: -- X-Spam_report: (-2.3 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.171, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1682709172566100004 Content-Type: text/plain; charset="utf-8" There is already include/qemu/stats.h, so stats.h was a bad idea. We want this file to not depend on anything else, we will move all the migration counters/stats to this struct. Signed-off-by: Juan Quintela Reviewed-by: Lukas Straub --- migration/meson.build | 1 + migration/migration-stats.c | 17 +++++++++++++++ migration/migration-stats.h | 41 +++++++++++++++++++++++++++++++++++++ migration/migration.c | 1 + migration/multifd.c | 1 + migration/ram.c | 3 +-- migration/ram.h | 23 --------------------- migration/savevm.c | 1 + 8 files changed, 63 insertions(+), 25 deletions(-) create mode 100644 migration/migration-stats.c create mode 100644 migration/migration-stats.h diff --git a/migration/meson.build b/migration/meson.build index 480ff6854a..da1897fadf 100644 --- a/migration/meson.build +++ b/migration/meson.build @@ -19,6 +19,7 @@ softmmu_ss.add(files( 'fd.c', 'global_state.c', 'migration-hmp-cmds.c', + 'migration-stats.c', 'migration.c', 'multifd.c', 'multifd-zlib.c', diff --git a/migration/migration-stats.c b/migration/migration-stats.c new file mode 100644 index 0000000000..b0eb5ae73c --- /dev/null +++ b/migration/migration-stats.c @@ -0,0 +1,17 @@ +/* + * Migration stats + * + * Copyright (c) 2012-2023 Red Hat Inc + * + * Authors: + * Juan Quintela + * + * This work is licensed under the terms of the GNU GPL, version 2 or late= r. + * See the COPYING file in the top-level directory. + */ + +#include "qemu/osdep.h" +#include "qemu/stats64.h" +#include "migration-stats.h" + +RAMStats ram_counters; diff --git a/migration/migration-stats.h b/migration/migration-stats.h new file mode 100644 index 0000000000..2edea0c779 --- /dev/null +++ b/migration/migration-stats.h @@ -0,0 +1,41 @@ +/* + * Migration stats + * + * Copyright (c) 2012-2023 Red Hat Inc + * + * Authors: + * Juan Quintela + * + * This work is licensed under the terms of the GNU GPL, version 2 or late= r. + * See the COPYING file in the top-level directory. + */ + +#ifndef QEMU_MIGRATION_STATS_H +#define QEMU_MIGRATION_STATS_H + +#include "qemu/stats64.h" + +/* + * These are the ram migration statistic counters. It is loosely + * based on MigrationStats. We change to Stat64 any counter that + * needs to be updated using atomic ops (can be accessed by more than + * one thread). + */ +typedef struct { + Stat64 dirty_bytes_last_sync; + Stat64 dirty_pages_rate; + Stat64 dirty_sync_count; + Stat64 dirty_sync_missed_zero_copy; + Stat64 downtime_bytes; + Stat64 zero_pages; + Stat64 multifd_bytes; + Stat64 normal_pages; + Stat64 postcopy_bytes; + Stat64 postcopy_requests; + Stat64 precopy_bytes; + Stat64 transferred; +} RAMStats; + +extern RAMStats ram_counters; + +#endif diff --git a/migration/migration.c b/migration/migration.c index abcadbb619..5ecf3dc381 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -29,6 +29,7 @@ #include "migration/global_state.h" #include "migration/misc.h" #include "migration.h" +#include "migration-stats.h" #include "savevm.h" #include "qemu-file.h" #include "channel.h" diff --git a/migration/multifd.c b/migration/multifd.c index 6053012ad9..347999f84a 100644 --- a/migration/multifd.c +++ b/migration/multifd.c @@ -19,6 +19,7 @@ #include "qapi/error.h" #include "ram.h" #include "migration.h" +#include "migration-stats.h" #include "socket.h" #include "tls.h" #include "qemu-file.h" diff --git a/migration/ram.c b/migration/ram.c index 89be3e3320..a6d5478ef8 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -36,6 +36,7 @@ #include "xbzrle.h" #include "ram.h" #include "migration.h" +#include "migration-stats.h" #include "migration/register.h" #include "migration/misc.h" #include "qemu-file.h" @@ -460,8 +461,6 @@ uint64_t ram_bytes_remaining(void) 0; } =20 -RAMStats ram_counters; - void ram_transferred_add(uint64_t bytes) { if (runstate_is_running()) { diff --git a/migration/ram.h b/migration/ram.h index 04b05e1b2c..8692de6ba0 100644 --- a/migration/ram.h +++ b/migration/ram.h @@ -32,30 +32,7 @@ #include "qapi/qapi-types-migration.h" #include "exec/cpu-common.h" #include "io/channel.h" -#include "qemu/stats64.h" =20 -/* - * These are the ram migration statistic counters. It is loosely - * based on MigrationStats. We change to Stat64 any counter that - * needs to be updated using atomic ops (can be accessed by more than - * one thread). - */ -typedef struct { - Stat64 dirty_bytes_last_sync; - Stat64 dirty_pages_rate; - Stat64 dirty_sync_count; - Stat64 dirty_sync_missed_zero_copy; - Stat64 downtime_bytes; - Stat64 zero_pages; - Stat64 multifd_bytes; - Stat64 normal_pages; - Stat64 postcopy_bytes; - Stat64 postcopy_requests; - Stat64 precopy_bytes; - Stat64 transferred; -} RAMStats; - -extern RAMStats ram_counters; extern XBZRLECacheStats xbzrle_counters; extern CompressionStats compression_counters; =20 diff --git a/migration/savevm.c b/migration/savevm.c index a9181b444b..8e2efb1a19 100644 --- a/migration/savevm.c +++ b/migration/savevm.c @@ -31,6 +31,7 @@ #include "net/net.h" #include "migration.h" #include "migration/snapshot.h" +#include "migration-stats.h" #include "migration/vmstate.h" #include "migration/misc.h" #include "migration/register.h" --=20 2.40.0 From nobody Fri May 17 02:03:38 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1682709171; cv=none; d=zohomail.com; s=zohoarc; b=TnhvAFnFuh+7Vngq5uqDRrzBgdgLsmNtcV8VgMX4eVjMrE/1QKjtvNNsBGzr9oSdXDYKKWqO1OE7Tk+Cp1n1XvoldYwkiMBOh1tS2qL0VtSW9Y2qViOav5BeInu61Dba5OqSzWS7LCnhpZAegaYifB20yDYjlwV+uyXFC1CSP8U= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1682709171; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=fHBO4s/UNvKYohfBJJwq43wMwIPGWXuavyCOYVzB90U=; b=e0Ho6i+kfdG9T/7W86ADhEsYariKcbznxCwO+49QxPIRXsXAYpBFrgXm8haXsBbTz4ASEJek22TAwYWOGbhcJebfFLtt2eCucZDTPcFhtH9lslHpCZBO7dHoXpe0aJXBtUKBNVcmC2ubqZeonge8GsJGzxNUeoUMy9K9QtEG06c= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 168270917123829.293600644541925; Fri, 28 Apr 2023 12:12:51 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1psTW9-0004jK-1H; Fri, 28 Apr 2023 15:12:29 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1psTW2-0004dM-9M for qemu-devel@nongnu.org; Fri, 28 Apr 2023 15:12:23 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1psTVy-0002vB-Nh for qemu-devel@nongnu.org; Fri, 28 Apr 2023 15:12:21 -0400 Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-119-gGu05iJ9MgOGgzCJds2_Mg-1; Fri, 28 Apr 2023 15:12:11 -0400 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.rdu2.redhat.com [10.11.54.6]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id D3199380662A; Fri, 28 Apr 2023 19:12:10 +0000 (UTC) Received: from secure.mitica (unknown [10.39.193.238]) by smtp.corp.redhat.com (Postfix) with ESMTP id 6437E2166B4F; Fri, 28 Apr 2023 19:12:09 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1682709137; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=fHBO4s/UNvKYohfBJJwq43wMwIPGWXuavyCOYVzB90U=; b=XIi14fotNTyxY+fiwzysMWi3h/K+HwRum0yjYdyVGh/3Sabnkt09mujSEaoIXyCvVSzwaA yPH2lvrxQKb5+bt2pvc3JEkd0knxPEuqM2hTk8LQD4AJmaaHE3DRu1H4wou2z7GWIBX3ds 3CfCKFi0RGoMhevVTowIg/n0qpUmJhk= X-MC-Unique: gGu05iJ9MgOGgzCJds2_Mg-1 From: Juan Quintela To: qemu-devel@nongnu.org Cc: Leonardo Bras , Juan Quintela , Thomas Huth , Laurent Vivier , Paolo Bonzini , Peter Xu , Lukas Straub Subject: [PULL 03/21] migration: Rename ram_counters to mig_stats Date: Fri, 28 Apr 2023 21:11:45 +0200 Message-Id: <20230428191203.39520-4-quintela@redhat.com> In-Reply-To: <20230428191203.39520-1-quintela@redhat.com> References: <20230428191203.39520-1-quintela@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.1 on 10.11.54.6 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=quintela@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -22 X-Spam_score: -2.3 X-Spam_bar: -- X-Spam_report: (-2.3 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.171, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1682709173148100012 Content-Type: text/plain; charset="utf-8" migration_stats is just too long, and it is going to have more than ram counters in the near future. Signed-off-by: Juan Quintela Reviewed-by: Lukas Straub --- migration/migration-stats.c | 2 +- migration/migration-stats.h | 2 +- migration/migration.c | 32 ++++++++++++------------- migration/multifd.c | 6 ++--- migration/ram.c | 48 ++++++++++++++++++------------------- migration/savevm.c | 2 +- 6 files changed, 46 insertions(+), 46 deletions(-) diff --git a/migration/migration-stats.c b/migration/migration-stats.c index b0eb5ae73c..8c0af9b80a 100644 --- a/migration/migration-stats.c +++ b/migration/migration-stats.c @@ -14,4 +14,4 @@ #include "qemu/stats64.h" #include "migration-stats.h" =20 -RAMStats ram_counters; +RAMStats mig_stats; diff --git a/migration/migration-stats.h b/migration/migration-stats.h index 2edea0c779..197374b4f6 100644 --- a/migration/migration-stats.h +++ b/migration/migration-stats.h @@ -36,6 +36,6 @@ typedef struct { Stat64 transferred; } RAMStats; =20 -extern RAMStats ram_counters; +extern RAMStats mig_stats; =20 #endif diff --git a/migration/migration.c b/migration/migration.c index 5ecf3dc381..feb5ab7493 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -909,26 +909,26 @@ static void populate_ram_info(MigrationInfo *info, Mi= grationState *s) size_t page_size =3D qemu_target_page_size(); =20 info->ram =3D g_malloc0(sizeof(*info->ram)); - info->ram->transferred =3D stat64_get(&ram_counters.transferred); + info->ram->transferred =3D stat64_get(&mig_stats.transferred); info->ram->total =3D ram_bytes_total(); - info->ram->duplicate =3D stat64_get(&ram_counters.zero_pages); + info->ram->duplicate =3D stat64_get(&mig_stats.zero_pages); /* legacy value. It is not used anymore */ info->ram->skipped =3D 0; - info->ram->normal =3D stat64_get(&ram_counters.normal_pages); + info->ram->normal =3D stat64_get(&mig_stats.normal_pages); info->ram->normal_bytes =3D info->ram->normal * page_size; info->ram->mbps =3D s->mbps; info->ram->dirty_sync_count =3D - stat64_get(&ram_counters.dirty_sync_count); + stat64_get(&mig_stats.dirty_sync_count); info->ram->dirty_sync_missed_zero_copy =3D - stat64_get(&ram_counters.dirty_sync_missed_zero_copy); + stat64_get(&mig_stats.dirty_sync_missed_zero_copy); info->ram->postcopy_requests =3D - stat64_get(&ram_counters.postcopy_requests); + stat64_get(&mig_stats.postcopy_requests); info->ram->page_size =3D page_size; - info->ram->multifd_bytes =3D stat64_get(&ram_counters.multifd_bytes); + info->ram->multifd_bytes =3D stat64_get(&mig_stats.multifd_bytes); info->ram->pages_per_second =3D s->pages_per_second; - info->ram->precopy_bytes =3D stat64_get(&ram_counters.precopy_bytes); - info->ram->downtime_bytes =3D stat64_get(&ram_counters.downtime_bytes); - info->ram->postcopy_bytes =3D stat64_get(&ram_counters.postcopy_bytes); + info->ram->precopy_bytes =3D stat64_get(&mig_stats.precopy_bytes); + info->ram->downtime_bytes =3D stat64_get(&mig_stats.downtime_bytes); + info->ram->postcopy_bytes =3D stat64_get(&mig_stats.postcopy_bytes); =20 if (migrate_xbzrle()) { info->xbzrle_cache =3D g_malloc0(sizeof(*info->xbzrle_cache)); @@ -960,7 +960,7 @@ static void populate_ram_info(MigrationInfo *info, Migr= ationState *s) if (s->state !=3D MIGRATION_STATUS_COMPLETED) { info->ram->remaining =3D ram_bytes_remaining(); info->ram->dirty_pages_rate =3D - stat64_get(&ram_counters.dirty_pages_rate); + stat64_get(&mig_stats.dirty_pages_rate); } } =20 @@ -1613,10 +1613,10 @@ static bool migrate_prepare(MigrationState *s, bool= blk, bool blk_inc, =20 migrate_init(s); /* - * set ram_counters compression_counters memory to zero for a + * set mig_stats compression_counters memory to zero for a * new migration */ - memset(&ram_counters, 0, sizeof(ram_counters)); + memset(&mig_stats, 0, sizeof(mig_stats)); memset(&compression_counters, 0, sizeof(compression_counters)); =20 return true; @@ -2627,7 +2627,7 @@ static MigThrError migration_detect_error(MigrationSt= ate *s) static uint64_t migration_total_bytes(MigrationState *s) { return qemu_file_total_transferred(s->to_dst_file) + - stat64_get(&ram_counters.multifd_bytes); + stat64_get(&mig_stats.multifd_bytes); } =20 static void migration_calculate_complete(MigrationState *s) @@ -2691,10 +2691,10 @@ static void migration_update_counters(MigrationStat= e *s, * if we haven't sent anything, we don't want to * recalculate. 10000 is a small enough number for our purposes */ - if (stat64_get(&ram_counters.dirty_pages_rate) && + if (stat64_get(&mig_stats.dirty_pages_rate) && transferred > 10000) { s->expected_downtime =3D - stat64_get(&ram_counters.dirty_bytes_last_sync) / bandwidth; + stat64_get(&mig_stats.dirty_bytes_last_sync) / bandwidth; } =20 qemu_file_reset_rate_limit(s->to_dst_file); diff --git a/migration/multifd.c b/migration/multifd.c index 347999f84a..4a2e1a47ce 100644 --- a/migration/multifd.c +++ b/migration/multifd.c @@ -434,8 +434,8 @@ static int multifd_send_pages(QEMUFile *f) transferred =3D ((uint64_t) pages->num) * p->page_size + p->packet_len; qemu_file_acct_rate_limit(f, transferred); qemu_mutex_unlock(&p->mutex); - stat64_add(&ram_counters.transferred, transferred); - stat64_add(&ram_counters.multifd_bytes, transferred); + stat64_add(&mig_stats.transferred, transferred); + stat64_add(&mig_stats.multifd_bytes, transferred); qemu_sem_post(&p->sem); =20 return 1; @@ -577,7 +577,7 @@ static int multifd_zero_copy_flush(QIOChannel *c) return -1; } if (ret =3D=3D 1) { - stat64_add(&ram_counters.dirty_sync_missed_zero_copy, 1); + stat64_add(&mig_stats.dirty_sync_missed_zero_copy, 1); } =20 return ret; diff --git a/migration/ram.c b/migration/ram.c index a6d5478ef8..c3981f64e4 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -464,13 +464,13 @@ uint64_t ram_bytes_remaining(void) void ram_transferred_add(uint64_t bytes) { if (runstate_is_running()) { - stat64_add(&ram_counters.precopy_bytes, bytes); + stat64_add(&mig_stats.precopy_bytes, bytes); } else if (migration_in_postcopy()) { - stat64_add(&ram_counters.postcopy_bytes, bytes); + stat64_add(&mig_stats.postcopy_bytes, bytes); } else { - stat64_add(&ram_counters.downtime_bytes, bytes); + stat64_add(&mig_stats.downtime_bytes, bytes); } - stat64_add(&ram_counters.transferred, bytes); + stat64_add(&mig_stats.transferred, bytes); } =20 struct MigrationOps { @@ -744,7 +744,7 @@ void mig_throttle_counter_reset(void) =20 rs->time_last_bitmap_sync =3D qemu_clock_get_ms(QEMU_CLOCK_REALTIME); rs->num_dirty_pages_period =3D 0; - rs->bytes_xfer_prev =3D stat64_get(&ram_counters.transferred); + rs->bytes_xfer_prev =3D stat64_get(&mig_stats.transferred); } =20 /** @@ -764,7 +764,7 @@ static void xbzrle_cache_zero_page(RAMState *rs, ram_ad= dr_t current_addr) /* We don't care if this fails to allocate a new cache page * as long as it updated an old one */ cache_insert(XBZRLE.cache, current_addr, XBZRLE.zero_target_page, - stat64_get(&ram_counters.dirty_sync_count)); + stat64_get(&mig_stats.dirty_sync_count)); } =20 #define ENCODING_FLAG_XBZRLE 0x1 @@ -790,7 +790,7 @@ static int save_xbzrle_page(RAMState *rs, PageSearchSta= tus *pss, int encoded_len =3D 0, bytes_xbzrle; uint8_t *prev_cached_page; QEMUFile *file =3D pss->pss_channel; - uint64_t generation =3D stat64_get(&ram_counters.dirty_sync_count); + uint64_t generation =3D stat64_get(&mig_stats.dirty_sync_count); =20 if (!cache_is_cached(XBZRLE.cache, current_addr, generation)) { xbzrle_counters.cache_miss++; @@ -1118,8 +1118,8 @@ uint64_t ram_pagesize_summary(void) =20 uint64_t ram_get_total_transferred_pages(void) { - return stat64_get(&ram_counters.normal_pages) + - stat64_get(&ram_counters.zero_pages) + + return stat64_get(&mig_stats.normal_pages) + + stat64_get(&mig_stats.zero_pages) + compression_counters.pages + xbzrle_counters.pages; } =20 @@ -1129,7 +1129,7 @@ static void migration_update_rates(RAMState *rs, int6= 4_t end_time) double compressed_size; =20 /* calculate period counters */ - stat64_set(&ram_counters.dirty_pages_rate, + stat64_set(&mig_stats.dirty_pages_rate, rs->num_dirty_pages_period * 1000 / (end_time - rs->time_last_bitmap_sync)); =20 @@ -1180,7 +1180,7 @@ static void migration_trigger_throttle(RAMState *rs) { uint64_t threshold =3D migrate_throttle_trigger_threshold(); uint64_t bytes_xfer_period =3D - stat64_get(&ram_counters.transferred) - rs->bytes_xfer_prev; + stat64_get(&mig_stats.transferred) - rs->bytes_xfer_prev; uint64_t bytes_dirty_period =3D rs->num_dirty_pages_period * TARGET_PA= GE_SIZE; uint64_t bytes_dirty_threshold =3D bytes_xfer_period * threshold / 100; =20 @@ -1209,7 +1209,7 @@ static void migration_bitmap_sync(RAMState *rs) RAMBlock *block; int64_t end_time; =20 - stat64_add(&ram_counters.dirty_sync_count, 1); + stat64_add(&mig_stats.dirty_sync_count, 1); =20 if (!rs->time_last_bitmap_sync) { rs->time_last_bitmap_sync =3D qemu_clock_get_ms(QEMU_CLOCK_REALTIM= E); @@ -1223,7 +1223,7 @@ static void migration_bitmap_sync(RAMState *rs) RAMBLOCK_FOREACH_NOT_IGNORED(block) { ramblock_sync_dirty_bitmap(rs, block); } - stat64_set(&ram_counters.dirty_bytes_last_sync, ram_bytes_remainin= g()); + stat64_set(&mig_stats.dirty_bytes_last_sync, ram_bytes_remaining()= ); } qemu_mutex_unlock(&rs->bitmap_mutex); =20 @@ -1243,10 +1243,10 @@ static void migration_bitmap_sync(RAMState *rs) /* reset period counters */ rs->time_last_bitmap_sync =3D end_time; rs->num_dirty_pages_period =3D 0; - rs->bytes_xfer_prev =3D stat64_get(&ram_counters.transferred); + rs->bytes_xfer_prev =3D stat64_get(&mig_stats.transferred); } if (migrate_events()) { - uint64_t generation =3D stat64_get(&ram_counters.dirty_sync_count); + uint64_t generation =3D stat64_get(&mig_stats.dirty_sync_count); qapi_event_send_migration_pass(generation); } } @@ -1320,7 +1320,7 @@ static int save_zero_page(PageSearchStatus *pss, QEMU= File *f, RAMBlock *block, int len =3D save_zero_page_to_file(pss, f, block, offset); =20 if (len) { - stat64_add(&ram_counters.zero_pages, 1); + stat64_add(&mig_stats.zero_pages, 1); ram_transferred_add(len); return 1; } @@ -1357,9 +1357,9 @@ static bool control_save_page(PageSearchStatus *pss, = RAMBlock *block, } =20 if (bytes_xmit > 0) { - stat64_add(&ram_counters.normal_pages, 1); + stat64_add(&mig_stats.normal_pages, 1); } else if (bytes_xmit =3D=3D 0) { - stat64_add(&ram_counters.zero_pages, 1); + stat64_add(&mig_stats.zero_pages, 1); } =20 return true; @@ -1391,7 +1391,7 @@ static int save_normal_page(PageSearchStatus *pss, RA= MBlock *block, qemu_put_buffer(file, buf, TARGET_PAGE_SIZE); } ram_transferred_add(TARGET_PAGE_SIZE); - stat64_add(&ram_counters.normal_pages, 1); + stat64_add(&mig_stats.normal_pages, 1); return 1; } =20 @@ -1447,7 +1447,7 @@ static int ram_save_multifd_page(QEMUFile *file, RAMB= lock *block, if (multifd_queue_page(file, block, offset) < 0) { return -1; } - stat64_add(&ram_counters.normal_pages, 1); + stat64_add(&mig_stats.normal_pages, 1); =20 return 1; } @@ -1486,7 +1486,7 @@ update_compress_thread_counts(const CompressParam *pa= ram, int bytes_xmit) ram_transferred_add(bytes_xmit); =20 if (param->zero_page) { - stat64_add(&ram_counters.zero_pages, 1); + stat64_add(&mig_stats.zero_pages, 1); return; } =20 @@ -2179,7 +2179,7 @@ int ram_save_queue_pages(const char *rbname, ram_addr= _t start, ram_addr_t len) RAMBlock *ramblock; RAMState *rs =3D ram_state; =20 - stat64_add(&ram_counters.postcopy_requests, 1); + stat64_add(&mig_stats.postcopy_requests, 1); RCU_READ_LOCK_GUARD(); =20 if (!rbname) { @@ -2634,9 +2634,9 @@ void acct_update_position(QEMUFile *f, size_t size, b= ool zero) uint64_t pages =3D size / TARGET_PAGE_SIZE; =20 if (zero) { - stat64_add(&ram_counters.zero_pages, pages); + stat64_add(&mig_stats.zero_pages, pages); } else { - stat64_add(&ram_counters.normal_pages, pages); + stat64_add(&mig_stats.normal_pages, pages); ram_transferred_add(size); qemu_file_credit_transfer(f, size); } diff --git a/migration/savevm.c b/migration/savevm.c index 8e2efb1a19..a9d0a88e62 100644 --- a/migration/savevm.c +++ b/migration/savevm.c @@ -1622,7 +1622,7 @@ static int qemu_savevm_state(QEMUFile *f, Error **err= p) } =20 migrate_init(ms); - memset(&ram_counters, 0, sizeof(ram_counters)); + memset(&mig_stats, 0, sizeof(mig_stats)); memset(&compression_counters, 0, sizeof(compression_counters)); ms->to_dst_file =3D f; =20 --=20 2.40.0 From nobody Fri May 17 02:03:38 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1682709476; cv=none; d=zohomail.com; s=zohoarc; b=kfI1C7aSvHGKU78HT8kS/KdvX05M9rb5fgIPn/66aerC43q7vBNiOQxIM0yLZRffOp5HzBLgHs9hyJrrQdjUuY7Lq4vw8qMMuYF1Cnjy6RtB8GVZdVbv81uO6ZE6nG2/ukaO9dFFNmOWrHpLEJvgPN3Zg4O/GWC9p5JBOUreC7g= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1682709476; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=jfae15PDbIUOl2CyCIvk9wObK0g/RFtr2m1QrHA2W6Q=; b=aJ2sNtmTFLNjqHnv5nGYO9hXqOxZEX8fBzcTfkvjs5fxfY7M95iIhbFZkwRRpZIrzpTR9vl7z72Ix8tRC+pUAn+7nZnfDEgeZYpkmxe6zgVmvVWOtF2fmqj0lIQur2Ri/ra/GcUWZCPniin50TUtPtqBTHlvmsK7C83IH0AuKmY= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 168270947685312.696277728516407; Fri, 28 Apr 2023 12:17:56 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1psTW8-0004j4-Qc; Fri, 28 Apr 2023 15:12:28 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1psTW2-0004dN-AC for qemu-devel@nongnu.org; Fri, 28 Apr 2023 15:12:23 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1psTVw-0002v5-Sy for qemu-devel@nongnu.org; Fri, 28 Apr 2023 15:12:20 -0400 Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-467-59CWqN7jPRmv_r84gCVOog-1; Fri, 28 Apr 2023 15:12:12 -0400 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.rdu2.redhat.com [10.11.54.6]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 783408314E8; Fri, 28 Apr 2023 19:12:12 +0000 (UTC) Received: from secure.mitica (unknown [10.39.193.238]) by smtp.corp.redhat.com (Postfix) with ESMTP id 29C422166B4F; Fri, 28 Apr 2023 19:12:11 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1682709136; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=jfae15PDbIUOl2CyCIvk9wObK0g/RFtr2m1QrHA2W6Q=; b=hyyEYK0iEDRWZqxkIg1TgYvt2rOgMXSKpnL42t2Y+GC8IKLK1IasVayfvEVK2aElkHIeZg XLVpNsanOUepY7n5FvVIbLiudFq5RSDT4f4ANxEzTKWyD9KCAjZ8ESR5Qs++SrK6Ds9/DN Kc0LaJr6Bq176EGbtufdg9I0kTdS0O8= X-MC-Unique: 59CWqN7jPRmv_r84gCVOog-1 From: Juan Quintela To: qemu-devel@nongnu.org Cc: Leonardo Bras , Juan Quintela , Thomas Huth , Laurent Vivier , Paolo Bonzini , Peter Xu , Lukas Straub Subject: [PULL 04/21] migration: Rename RAMStats to MigrationAtomicStats Date: Fri, 28 Apr 2023 21:11:46 +0200 Message-Id: <20230428191203.39520-5-quintela@redhat.com> In-Reply-To: <20230428191203.39520-1-quintela@redhat.com> References: <20230428191203.39520-1-quintela@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.1 on 10.11.54.6 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=quintela@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -22 X-Spam_score: -2.3 X-Spam_bar: -- X-Spam_report: (-2.3 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.171, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1682709478887100003 Content-Type: text/plain; charset="utf-8" It is lousely based on MigrationStats, but that name is taken, so this is the best one that I came with. Signed-off-by: Juan Quintela Reviewed-by: Lukas Straub --- If you have any good suggestion for the name, I am all ears. --- migration/migration-stats.c | 2 +- migration/migration-stats.h | 4 ++-- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/migration/migration-stats.c b/migration/migration-stats.c index 8c0af9b80a..2f2cea965c 100644 --- a/migration/migration-stats.c +++ b/migration/migration-stats.c @@ -14,4 +14,4 @@ #include "qemu/stats64.h" #include "migration-stats.h" =20 -RAMStats mig_stats; +MigrationAtomicStats mig_stats; diff --git a/migration/migration-stats.h b/migration/migration-stats.h index 197374b4f6..149af932d7 100644 --- a/migration/migration-stats.h +++ b/migration/migration-stats.h @@ -34,8 +34,8 @@ typedef struct { Stat64 postcopy_requests; Stat64 precopy_bytes; Stat64 transferred; -} RAMStats; +} MigrationAtomicStats; =20 -extern RAMStats mig_stats; +extern MigrationAtomicStats mig_stats; =20 #endif --=20 2.40.0 From nobody Fri May 17 02:03:38 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1682709180; cv=none; d=zohomail.com; s=zohoarc; b=RqT31AmEUXF5SR0zV/oRsVQZS21OzStRXz4sDruSh+jcCecGcC/aruiMUyJxIIxP/UvnfNt6Hk2lqV4ZgGHxkR3ag4ZDaT6vhFg+6jg6GA+rmVmhZrD17LM0Y0l+gpLmhSpguWLXzHKR5klcuVDrewWl2bCjDOlRqA65HT0p6tw= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1682709180; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=DxWHGYZc3NZRKmU+RyiF9FLRsVkR3VPuMtGQEkyhY2k=; b=ZuYPWJi/T1bBYnBCf6tAxOnEB6BAQF44iEp+R19EHS8DqTSy+T7mzq6QFQII98oDvpwHqv3irxIi0Fzq0KJypQUNBqIONTOucFnYN3dnO2Y4oR4t62wHlxNlvio3OICAO4UbC6LXVYzHCVMTV13r0qtMazp/zpLejYeq89fWxSk= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1682709180181357.29445371803695; Fri, 28 Apr 2023 12:13:00 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1psTW8-0004iP-5G; Fri, 28 Apr 2023 15:12:28 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1psTW2-0004dL-8y for qemu-devel@nongnu.org; Fri, 28 Apr 2023 15:12:22 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1psTVy-0002vG-By for qemu-devel@nongnu.org; Fri, 28 Apr 2023 15:12:20 -0400 Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-127-Fu2GqE99Or2NtV7hNfgoow-1; Fri, 28 Apr 2023 15:12:14 -0400 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.rdu2.redhat.com [10.11.54.6]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 2D5A685530C; Fri, 28 Apr 2023 19:12:14 +0000 (UTC) Received: from secure.mitica (unknown [10.39.193.238]) by smtp.corp.redhat.com (Postfix) with ESMTP id BA80C2166B4F; Fri, 28 Apr 2023 19:12:12 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1682709137; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=DxWHGYZc3NZRKmU+RyiF9FLRsVkR3VPuMtGQEkyhY2k=; b=RKku2Aq1yG2TRYYi3v8A6A9a/BG0h7RDO8WCFYZxGfH6VeuER1asMDDbqb7VV3jQo4So63 3rJaKfvutoyOuT4UGu0kiCHjGDwuqy9xNjiqeNceh/GxO0lRROFqrFMqr9NhSXztNbYIVp NTuOYjPeOK+8csNpZHfQTvpdtVECOZQ= X-MC-Unique: Fu2GqE99Or2NtV7hNfgoow-1 From: Juan Quintela To: qemu-devel@nongnu.org Cc: Leonardo Bras , Juan Quintela , Thomas Huth , Laurent Vivier , Paolo Bonzini , Peter Xu , Lukas Straub Subject: [PULL 05/21] migration/rdma: Split the zero page case from acct_update_position Date: Fri, 28 Apr 2023 21:11:47 +0200 Message-Id: <20230428191203.39520-6-quintela@redhat.com> In-Reply-To: <20230428191203.39520-1-quintela@redhat.com> References: <20230428191203.39520-1-quintela@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.1 on 10.11.54.6 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=quintela@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -22 X-Spam_score: -2.3 X-Spam_bar: -- X-Spam_report: (-2.3 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.171, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1682709182024100015 Content-Type: text/plain; charset="utf-8" Now that we have atomic counters, we can do it on the place that we need it, no need to do it inside ram.c. Signed-off-by: Juan Quintela Reviewed-by: Lukas Straub --- migration/ram.c | 12 ++++-------- migration/ram.h | 2 +- migration/rdma.c | 7 +++++-- 3 files changed, 10 insertions(+), 11 deletions(-) diff --git a/migration/ram.c b/migration/ram.c index c3981f64e4..c249a1f468 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -2629,17 +2629,13 @@ static int ram_find_and_save_block(RAMState *rs) return pages; } =20 -void acct_update_position(QEMUFile *f, size_t size, bool zero) +void acct_update_position(QEMUFile *f, size_t size) { uint64_t pages =3D size / TARGET_PAGE_SIZE; =20 - if (zero) { - stat64_add(&mig_stats.zero_pages, pages); - } else { - stat64_add(&mig_stats.normal_pages, pages); - ram_transferred_add(size); - qemu_file_credit_transfer(f, size); - } + stat64_add(&mig_stats.normal_pages, pages); + ram_transferred_add(size); + qemu_file_credit_transfer(f, size); } =20 static uint64_t ram_bytes_total_with_ignored(void) diff --git a/migration/ram.h b/migration/ram.h index 8692de6ba0..3804753ca3 100644 --- a/migration/ram.h +++ b/migration/ram.h @@ -53,7 +53,7 @@ void mig_throttle_counter_reset(void); =20 uint64_t ram_pagesize_summary(void); int ram_save_queue_pages(const char *rbname, ram_addr_t start, ram_addr_t = len); -void acct_update_position(QEMUFile *f, size_t size, bool zero); +void acct_update_position(QEMUFile *f, size_t size); void ram_postcopy_migrated_memory_release(MigrationState *ms); /* For outgoing discard bitmap */ void ram_postcopy_send_discard_bitmap(MigrationState *ms); diff --git a/migration/rdma.c b/migration/rdma.c index 0af5e944f0..7a9b284c3f 100644 --- a/migration/rdma.c +++ b/migration/rdma.c @@ -17,8 +17,10 @@ #include "qemu/osdep.h" #include "qapi/error.h" #include "qemu/cutils.h" +#include "exec/target_page.h" #include "rdma.h" #include "migration.h" +#include "migration-stats.h" #include "qemu-file.h" #include "ram.h" #include "qemu/error-report.h" @@ -2120,7 +2122,8 @@ retry: return -EIO; } =20 - acct_update_position(f, sge.length, true); + stat64_add(&mig_stats.zero_pages, + sge.length / qemu_target_page_size()); =20 return 1; } @@ -2228,7 +2231,7 @@ retry: } =20 set_bit(chunk, block->transit_bitmap); - acct_update_position(f, sge.length, false); + acct_update_position(f, sge.length); rdma->total_writes++; =20 return 0; --=20 2.40.0 From nobody Fri May 17 02:03:38 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1682709178; cv=none; d=zohomail.com; s=zohoarc; b=AFKxOHBUGncMIYmabeQuVVf+SfaH9uTTu1vaGWWD/ufMwFaHKyZQ2W15Op1OdWZ0wyf5Tk8jeOO2uYy5A40sibMa9anWASJpszRWZI/vjc/NT+h3j4IoksJ4AcqGefNugPsVCZhzPNA+0eC//SY79KvBCoxm4W911aHcFX7jIzY= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1682709178; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=l38tWqn5Fu5lrmauLijpLYzD9pCl3Gw0p0V7GhyMam8=; b=GLCbiM4C3fTJMK5IhnV2pvCSOvsDzVtdFonMnrg/9OmY/x71Sgh11+EbB7O1eaPPdi7F9DNAssxvWuvGjfCmPyvlDF+1OWU3V4xdZOv3QSR8OBWW49i/gb4TUTjAUc18tFRiNhOKSwe+12+rtbCcVV7mfMgJL6AQlozvLhwlpY0= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1682709178598492.06672929062574; Fri, 28 Apr 2023 12:12:58 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1psTW9-0004kC-DZ; Fri, 28 Apr 2023 15:12:29 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1psTW7-0004hg-Uu for qemu-devel@nongnu.org; Fri, 28 Apr 2023 15:12:27 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1psTW1-0002vu-UY for qemu-devel@nongnu.org; Fri, 28 Apr 2023 15:12:24 -0400 Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-52-Mg7PsS7zPg68PYi-_VxlMg-1; Fri, 28 Apr 2023 15:12:16 -0400 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.rdu2.redhat.com [10.11.54.6]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id D48C9185A7A2; Fri, 28 Apr 2023 19:12:15 +0000 (UTC) Received: from secure.mitica (unknown [10.39.193.238]) by smtp.corp.redhat.com (Postfix) with ESMTP id 71DFA2166B4F; Fri, 28 Apr 2023 19:12:14 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1682709139; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=l38tWqn5Fu5lrmauLijpLYzD9pCl3Gw0p0V7GhyMam8=; b=cgaijLEAhJ99KlI+zAaKnP6YpfwJCQkqqP12pZfzEJ8yShIK367ukCo0Yb1LxtiU+wMNqk E8Un4XizGXApudLt94s8KAmv7VKIejrMlz7kwsXlxY/uQMP1y4ddBsIOGViS5g7uPbZT8x KrHg/nQ9XAgl2LanotXUyfWxTxv4iAs= X-MC-Unique: Mg7PsS7zPg68PYi-_VxlMg-1 From: Juan Quintela To: qemu-devel@nongnu.org Cc: Leonardo Bras , Juan Quintela , Thomas Huth , Laurent Vivier , Paolo Bonzini , Peter Xu , Lukas Straub Subject: [PULL 06/21] migration/rdma: Unfold last user of acct_update_position() Date: Fri, 28 Apr 2023 21:11:48 +0200 Message-Id: <20230428191203.39520-7-quintela@redhat.com> In-Reply-To: <20230428191203.39520-1-quintela@redhat.com> References: <20230428191203.39520-1-quintela@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.1 on 10.11.54.6 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=quintela@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -22 X-Spam_score: -2.3 X-Spam_bar: -- X-Spam_report: (-2.3 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.171, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1682709178971100001 Content-Type: text/plain; charset="utf-8" Signed-off-by: Juan Quintela Reviewed-by: Lukas Straub --- migration/ram.c | 9 --------- migration/ram.h | 1 - migration/rdma.c | 4 +++- 3 files changed, 3 insertions(+), 11 deletions(-) diff --git a/migration/ram.c b/migration/ram.c index c249a1f468..7d81c4a39e 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -2629,15 +2629,6 @@ static int ram_find_and_save_block(RAMState *rs) return pages; } =20 -void acct_update_position(QEMUFile *f, size_t size) -{ - uint64_t pages =3D size / TARGET_PAGE_SIZE; - - stat64_add(&mig_stats.normal_pages, pages); - ram_transferred_add(size); - qemu_file_credit_transfer(f, size); -} - static uint64_t ram_bytes_total_with_ignored(void) { RAMBlock *block; diff --git a/migration/ram.h b/migration/ram.h index 3804753ca3..6fffbeb5f1 100644 --- a/migration/ram.h +++ b/migration/ram.h @@ -53,7 +53,6 @@ void mig_throttle_counter_reset(void); =20 uint64_t ram_pagesize_summary(void); int ram_save_queue_pages(const char *rbname, ram_addr_t start, ram_addr_t = len); -void acct_update_position(QEMUFile *f, size_t size); void ram_postcopy_migrated_memory_release(MigrationState *ms); /* For outgoing discard bitmap */ void ram_postcopy_send_discard_bitmap(MigrationState *ms); diff --git a/migration/rdma.c b/migration/rdma.c index 7a9b284c3f..7e747b2595 100644 --- a/migration/rdma.c +++ b/migration/rdma.c @@ -2231,7 +2231,9 @@ retry: } =20 set_bit(chunk, block->transit_bitmap); - acct_update_position(f, sge.length); + stat64_add(&mig_stats.normal_pages, sge.length / qemu_target_page_size= ()); + ram_transferred_add(sge.length); + qemu_file_credit_transfer(f, sge.length); rdma->total_writes++; =20 return 0; --=20 2.40.0 From nobody Fri May 17 02:03:38 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1682709299; cv=none; d=zohomail.com; s=zohoarc; b=LTWyhdLyLfteBMh1H4eyYZLuFJRAyT52PWg5WKpiTizwJFq2yYt0DTu2ei7R7JGkhvt6gSVHdWcb6JA37I5zUD0KQDE259Z8tEU9jjqYPx0jFBCa6I/JusHH4H0apd/60T65kxiRp9M3JjyiZwCDYhwj7gzU0w19eYR5gqHyIz0= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1682709299; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=bezRbyElX4oDQkDmYg3oZ+p/BWTANa/wf2MhX9HYs/8=; b=LOt9xQTeoYpzKk1N0Yj0GgyWR3XNbJkm3F8VIMi5VqcUscSLF2hcjYB9XPb3/9KVwGLVy+XE5n5Z9uUpBrHyH1W387Khx8YTCfulwW1KH3cYxZaom14sQ6i8aKRyMDv5Xhz0VbdtwG1+v5EvTW99l1Rpuiiy/AF/pUor0L8kQJ0= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1682709299554521.0477945815724; Fri, 28 Apr 2023 12:14:59 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1psTW9-0004kt-Rw; Fri, 28 Apr 2023 15:12:30 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1psTW7-0004hb-T2 for qemu-devel@nongnu.org; Fri, 28 Apr 2023 15:12:27 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1psTW1-0002xQ-UR for qemu-devel@nongnu.org; Fri, 28 Apr 2023 15:12:23 -0400 Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-352-nKjCn_dvNTS32UJ9qXZqQQ-1; Fri, 28 Apr 2023 15:12:18 -0400 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.rdu2.redhat.com [10.11.54.6]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 8E2E7185A790; Fri, 28 Apr 2023 19:12:17 +0000 (UTC) Received: from secure.mitica (unknown [10.39.193.238]) by smtp.corp.redhat.com (Postfix) with ESMTP id 22F7D2166B4F; Fri, 28 Apr 2023 19:12:16 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1682709140; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=bezRbyElX4oDQkDmYg3oZ+p/BWTANa/wf2MhX9HYs/8=; b=GHM2szPECWyTiwZkP4ZhPDM/EucWg+bdiCQf7ZwcQboLSUSyYxeixD3QLlvXToM/t7B+Qk jXHxp/16nKEv1mNEPoBbn0HuZtrjz8mh+TlQ1hZXCIA46JkMx+nV8OX6aBIxDEiWehNKwq baL+C7VylMK4nv8875lLx3Bk3LSjFW8= X-MC-Unique: nKjCn_dvNTS32UJ9qXZqQQ-1 From: Juan Quintela To: qemu-devel@nongnu.org Cc: Leonardo Bras , Juan Quintela , Thomas Huth , Laurent Vivier , Paolo Bonzini , Peter Xu , Vladimir Sementsov-Ogievskiy Subject: [PULL 07/21] migration: Drop unused parameter for migration_tls_get_creds() Date: Fri, 28 Apr 2023 21:11:49 +0200 Message-Id: <20230428191203.39520-8-quintela@redhat.com> In-Reply-To: <20230428191203.39520-1-quintela@redhat.com> References: <20230428191203.39520-1-quintela@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.1 on 10.11.54.6 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=quintela@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -22 X-Spam_score: -2.3 X-Spam_bar: -- X-Spam_report: (-2.3 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.171, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1682709300934100003 Content-Type: text/plain; charset="utf-8" It is not needed since we moved the accessor for tls properties to options.c. Suggested-by: Vladimir Sementsov-Ogievskiy Signed-off-by: Juan Quintela Reviewed-by: Vladimir Sementsov-Ogievskiy --- migration/tls.c | 10 +++------- 1 file changed, 3 insertions(+), 7 deletions(-) diff --git a/migration/tls.c b/migration/tls.c index cd29177957..d4a76cf590 100644 --- a/migration/tls.c +++ b/migration/tls.c @@ -29,9 +29,7 @@ #include "trace.h" =20 static QCryptoTLSCreds * -migration_tls_get_creds(MigrationState *s, - QCryptoTLSCredsEndpoint endpoint, - Error **errp) +migration_tls_get_creds(QCryptoTLSCredsEndpoint endpoint, Error **errp) { Object *creds; const char *tls_creds =3D migrate_tls_creds(); @@ -80,8 +78,7 @@ void migration_tls_channel_process_incoming(MigrationStat= e *s, QCryptoTLSCreds *creds; QIOChannelTLS *tioc; =20 - creds =3D migration_tls_get_creds( - s, QCRYPTO_TLS_CREDS_ENDPOINT_SERVER, errp); + creds =3D migration_tls_get_creds(QCRYPTO_TLS_CREDS_ENDPOINT_SERVER, e= rrp); if (!creds) { return; } @@ -124,8 +121,7 @@ QIOChannelTLS *migration_tls_client_create(MigrationSta= te *s, { QCryptoTLSCreds *creds; =20 - creds =3D migration_tls_get_creds( - s, QCRYPTO_TLS_CREDS_ENDPOINT_CLIENT, errp); + creds =3D migration_tls_get_creds(QCRYPTO_TLS_CREDS_ENDPOINT_CLIENT, e= rrp); if (!creds) { return NULL; } --=20 2.40.0 From nobody Fri May 17 02:03:38 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1682709396; cv=none; d=zohomail.com; s=zohoarc; b=OK3Ir+GHXaUOKY9Go8DQtOi4m79Ha4psNmHXQ3Sm2TJUkTHor1ZP5r+OBb7jQUvsSukBgTY0ZTQMI0+ZLV0M4UexyMxGTjpS6uXxblqbOINWe9+Y/+QOfIZaGzurL832KX9B8NT47t8Nta5nX21ZeHp8kL0UpLHqRl/xtJRDENk= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1682709396; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=E4P9/Comv2W+9H69u3LnQkSLu/Ua9tCZaXw7xoUFIfw=; b=NGTVDov6IQG2IQW+nUxV221WcGp3PwFvuKWZRuTKpY7pFVwzRLnXd+KIqGDh8ROEPKJmHrP9XzXqkq8VZJUELZzr+crmDRTDsi1dB40FVkcA0VbGX2auAbNJEBGszH/ZOG17Rznk7yVS4VaKfWAbAUxcbIJkVbTtHJzuDSsSv0U= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1682709396808754.3715311374375; Fri, 28 Apr 2023 12:16:36 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1psTWD-0004nG-MQ; Fri, 28 Apr 2023 15:12:33 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1psTW9-0004kB-Cs for qemu-devel@nongnu.org; Fri, 28 Apr 2023 15:12:29 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1psTW6-0002yC-03 for qemu-devel@nongnu.org; Fri, 28 Apr 2023 15:12:29 -0400 Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-380-1ZCv5NoJPLK9M_ybtHpZZA-1; Fri, 28 Apr 2023 15:12:20 -0400 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.rdu2.redhat.com [10.11.54.6]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id B4D65101A550; Fri, 28 Apr 2023 19:12:19 +0000 (UTC) Received: from secure.mitica (unknown [10.39.193.238]) by smtp.corp.redhat.com (Postfix) with ESMTP id D07F72166B4F; Fri, 28 Apr 2023 19:12:17 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1682709143; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=E4P9/Comv2W+9H69u3LnQkSLu/Ua9tCZaXw7xoUFIfw=; b=aUjrL8E0vtqGmi9h8U40ZMTqeX4RMZzxxtA/W9ylGTLoCB7QVD8wxBPN9ipyfAcuC+KIjd QiBILvn2iNfF5U/xIsyD8ILBmt9yj+b9WtweBxdxA8aBFJk+CD+l+5v8iOxqw0owLGdgM2 GHf1e5CtajsC08As8+5a7m/nVQ8O/gM= X-MC-Unique: 1ZCv5NoJPLK9M_ybtHpZZA-1 From: Juan Quintela To: qemu-devel@nongnu.org Cc: Leonardo Bras , Juan Quintela , Thomas Huth , Laurent Vivier , Paolo Bonzini , Peter Xu , Vladimir Sementsov-Ogievskiy Subject: [PULL 08/21] migration: Drop unused parameter for migration_tls_client_create() Date: Fri, 28 Apr 2023 21:11:50 +0200 Message-Id: <20230428191203.39520-9-quintela@redhat.com> In-Reply-To: <20230428191203.39520-1-quintela@redhat.com> References: <20230428191203.39520-1-quintela@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.1 on 10.11.54.6 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=quintela@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -22 X-Spam_score: -2.3 X-Spam_bar: -- X-Spam_report: (-2.3 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.171, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1682709398556100002 Content-Type: text/plain; charset="utf-8" It is not needed since we moved the accessor for tls properties to options.c. Suggested-by: Vladimir Sementsov-Ogievskiy Signed-off-by: Juan Quintela Reviewed-by: Vladimir Sementsov-Ogievskiy --- migration/multifd.c | 2 +- migration/postcopy-ram.c | 2 +- migration/tls.c | 5 ++--- migration/tls.h | 3 +-- 4 files changed, 5 insertions(+), 7 deletions(-) diff --git a/migration/multifd.c b/migration/multifd.c index 4a2e1a47ce..4e71c19292 100644 --- a/migration/multifd.c +++ b/migration/multifd.c @@ -821,7 +821,7 @@ static void multifd_tls_channel_connect(MultiFDSendPara= ms *p, const char *hostname =3D s->hostname; QIOChannelTLS *tioc; =20 - tioc =3D migration_tls_client_create(s, ioc, hostname, errp); + tioc =3D migration_tls_client_create(ioc, hostname, errp); if (!tioc) { return; } diff --git a/migration/postcopy-ram.c b/migration/postcopy-ram.c index 75aa276bb1..5615ec29eb 100644 --- a/migration/postcopy-ram.c +++ b/migration/postcopy-ram.c @@ -1632,7 +1632,7 @@ postcopy_preempt_send_channel_new(QIOTask *task, gpoi= nter opaque) } =20 if (migrate_channel_requires_tls_upgrade(ioc)) { - tioc =3D migration_tls_client_create(s, ioc, s->hostname, &local_e= rr); + tioc =3D migration_tls_client_create(ioc, s->hostname, &local_err); if (!tioc) { goto out; } diff --git a/migration/tls.c b/migration/tls.c index d4a76cf590..fa03d9136c 100644 --- a/migration/tls.c +++ b/migration/tls.c @@ -114,8 +114,7 @@ static void migration_tls_outgoing_handshake(QIOTask *t= ask, object_unref(OBJECT(ioc)); } =20 -QIOChannelTLS *migration_tls_client_create(MigrationState *s, - QIOChannel *ioc, +QIOChannelTLS *migration_tls_client_create(QIOChannel *ioc, const char *hostname, Error **errp) { @@ -141,7 +140,7 @@ void migration_tls_channel_connect(MigrationState *s, { QIOChannelTLS *tioc; =20 - tioc =3D migration_tls_client_create(s, ioc, hostname, errp); + tioc =3D migration_tls_client_create(ioc, hostname, errp); if (!tioc) { return; } diff --git a/migration/tls.h b/migration/tls.h index 98e23c9b0e..5797d153cb 100644 --- a/migration/tls.h +++ b/migration/tls.h @@ -28,8 +28,7 @@ void migration_tls_channel_process_incoming(MigrationStat= e *s, QIOChannel *ioc, Error **errp); =20 -QIOChannelTLS *migration_tls_client_create(MigrationState *s, - QIOChannel *ioc, +QIOChannelTLS *migration_tls_client_create(QIOChannel *ioc, const char *hostname, Error **errp); =20 --=20 2.40.0 From nobody Fri May 17 02:03:38 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1682709171; cv=none; d=zohomail.com; s=zohoarc; b=jnRe7KLQdiV5xQCdd3gwgFn99DPJKZQqa0+dZWWV+F8/zvJk2zVUL6rQ5bJpwkinAxtA8MQ0j3SevgL1VOmvjiijrIXjAel4davSGQfMrnWglLFcgNNW3+d1xvH5iHQn5BxBtFvUhAukz+Td1GdPyYYHRlxqmYA5Swf8KdJg0IY= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1682709171; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=z5BOUcS3NEKS3ep+Gx/z01Redsa8AlMTbGnP6mbzpWo=; b=EsThvCR8n1o6yv50OWcZPXZ1lmudeOCJU49CIGg7irQ0R0Dl2DacWGWNYshefLTMkofL5JuAzuP3tmkB7cA/dR0xjmKSPKCLXpA2xNHUXrtjkcc4cskGHFe3rSFvbCOZg8zbYXSKCcHChyCN9lYPp4mtliEfzeEIGrz/XRj8kpI= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1682709171291847.5299007077751; Fri, 28 Apr 2023 12:12:51 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1psTWF-0004pi-Oa; Fri, 28 Apr 2023 15:12:35 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1psTWC-0004mL-VK for qemu-devel@nongnu.org; Fri, 28 Apr 2023 15:12:32 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1psTW4-0002y8-Ng for qemu-devel@nongnu.org; Fri, 28 Apr 2023 15:12:30 -0400 Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-74-NYd30KgKNvaCcpf8A0J0gg-1; Fri, 28 Apr 2023 15:12:21 -0400 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.rdu2.redhat.com [10.11.54.6]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 7A4CA85A5A3; Fri, 28 Apr 2023 19:12:21 +0000 (UTC) Received: from secure.mitica (unknown [10.39.193.238]) by smtp.corp.redhat.com (Postfix) with ESMTP id 0B9312166B4F; Fri, 28 Apr 2023 19:12:19 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1682709143; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=z5BOUcS3NEKS3ep+Gx/z01Redsa8AlMTbGnP6mbzpWo=; b=a+5jomdei/rSmhgVulXNhKK/CYaM6p7r2wuyrSoE29sYiU9ZYQWidGLC6JYOcCtaUgjGgR OEfYSpzS/lUjAwfFEw5dYltWu3FUX65m5ahtHvBsQ1LbhVT3bupbLA3kLwSZdO7SZ1uNK1 Ycsjm0+1Cb8k6UOSQ03y8g0sbxbxpwY= X-MC-Unique: NYd30KgKNvaCcpf8A0J0gg-1 From: Juan Quintela To: qemu-devel@nongnu.org Cc: Leonardo Bras , Juan Quintela , Thomas Huth , Laurent Vivier , Paolo Bonzini , Peter Xu , Lukas Straub Subject: [PULL 09/21] qtest/migration-test.c: Add tests with compress enabled Date: Fri, 28 Apr 2023 21:11:51 +0200 Message-Id: <20230428191203.39520-10-quintela@redhat.com> In-Reply-To: <20230428191203.39520-1-quintela@redhat.com> References: <20230428191203.39520-1-quintela@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.1 on 10.11.54.6 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=quintela@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -22 X-Spam_score: -2.3 X-Spam_bar: -- X-Spam_report: (-2.3 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.171, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1682709172560100003 Content-Type: text/plain; charset="utf-8" From: Lukas Straub There has never been tests for migration with compress enabled. Add suitable tests, testing with compress-wait-thread =3D false too. Signed-off-by: Lukas Straub Acked-by: Peter Xu Reviewed-by: Juan Quintela Signed-off-by: Juan Quintela --- tests/qtest/migration-test.c | 103 +++++++++++++++++++++++++++++++++++ 1 file changed, 103 insertions(+) diff --git a/tests/qtest/migration-test.c b/tests/qtest/migration-test.c index 60dd53d3ec..d28b0d9880 100644 --- a/tests/qtest/migration-test.c +++ b/tests/qtest/migration-test.c @@ -406,6 +406,41 @@ static void migrate_set_parameter_str(QTestState *who,= const char *parameter, migrate_check_parameter_str(who, parameter, value); } =20 +static long long migrate_get_parameter_bool(QTestState *who, + const char *parameter) +{ + QDict *rsp; + int result; + + rsp =3D wait_command(who, "{ 'execute': 'query-migrate-parameters' }"); + result =3D qdict_get_bool(rsp, parameter); + qobject_unref(rsp); + return !!result; +} + +static void migrate_check_parameter_bool(QTestState *who, const char *para= meter, + int value) +{ + int result; + + result =3D migrate_get_parameter_bool(who, parameter); + g_assert_cmpint(result, =3D=3D, value); +} + +static void migrate_set_parameter_bool(QTestState *who, const char *parame= ter, + int value) +{ + QDict *rsp; + + rsp =3D qtest_qmp(who, + "{ 'execute': 'migrate-set-parameters'," + "'arguments': { %s: %i } }", + parameter, value); + g_assert(qdict_haskey(rsp, "return")); + qobject_unref(rsp); + migrate_check_parameter_bool(who, parameter, value); +} + static void migrate_ensure_non_converge(QTestState *who) { /* Can't converge with 1ms downtime + 3 mbs bandwidth limit */ @@ -1524,6 +1559,70 @@ static void test_precopy_unix_xbzrle(void) test_precopy_common(&args); } =20 +static void * +test_migrate_compress_start(QTestState *from, + QTestState *to) +{ + migrate_set_parameter_int(from, "compress-level", 1); + migrate_set_parameter_int(from, "compress-threads", 4); + migrate_set_parameter_bool(from, "compress-wait-thread", true); + migrate_set_parameter_int(to, "decompress-threads", 4); + + migrate_set_capability(from, "compress", true); + migrate_set_capability(to, "compress", true); + + return NULL; +} + +static void test_precopy_unix_compress(void) +{ + g_autofree char *uri =3D g_strdup_printf("unix:%s/migsocket", tmpfs); + MigrateCommon args =3D { + .connect_uri =3D uri, + .listen_uri =3D uri, + .start_hook =3D test_migrate_compress_start, + /* + * Test that no invalid thread state is left over from + * the previous iteration. + */ + .iterations =3D 2, + }; + + test_precopy_common(&args); +} + +static void * +test_migrate_compress_nowait_start(QTestState *from, + QTestState *to) +{ + migrate_set_parameter_int(from, "compress-level", 9); + migrate_set_parameter_int(from, "compress-threads", 1); + migrate_set_parameter_bool(from, "compress-wait-thread", false); + migrate_set_parameter_int(to, "decompress-threads", 1); + + migrate_set_capability(from, "compress", true); + migrate_set_capability(to, "compress", true); + + return NULL; +} + +static void test_precopy_unix_compress_nowait(void) +{ + g_autofree char *uri =3D g_strdup_printf("unix:%s/migsocket", tmpfs); + MigrateCommon args =3D { + .connect_uri =3D uri, + .listen_uri =3D uri, + .start_hook =3D test_migrate_compress_nowait_start, + /* + * Test that no invalid thread state is left over from + * the previous iteration. + */ + .iterations =3D 2, + }; + + test_precopy_common(&args); +} + static void test_precopy_tcp_plain(void) { MigrateCommon args =3D { @@ -2530,6 +2629,10 @@ int main(int argc, char **argv) qtest_add_func("/migration/bad_dest", test_baddest); qtest_add_func("/migration/precopy/unix/plain", test_precopy_unix_plai= n); qtest_add_func("/migration/precopy/unix/xbzrle", test_precopy_unix_xbz= rle); + qtest_add_func("/migration/precopy/unix/compress/wait", + test_precopy_unix_compress); + qtest_add_func("/migration/precopy/unix/compress/nowait", + test_precopy_unix_compress_nowait); #ifdef CONFIG_GNUTLS qtest_add_func("/migration/precopy/unix/tls/psk", test_precopy_unix_tls_psk); --=20 2.40.0 From nobody Fri May 17 02:03:38 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1682709178; cv=none; d=zohomail.com; s=zohoarc; b=F1W4aeVZ7J86ytDFFjQzx4P1w9AuR0EZ2w7joeKH3RyB90lmT7nd//jbjDiErN/qcLZMoupaJYR6VTEQzz8rXWdROalXWxNKiLrt26nT7IZh4BnxVHXaAASoUvENRaevsr+B00U7YUBx0mTEZ+vE4zk6YrfgqXcpKxnm+w4hTeg= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1682709178; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=RWuMgvR3+lr0+73m6jOfgll5e5994spFbRjX06qZlj4=; b=i/DpJw/GNEhpQG8/AvbcqmCYqSj7QgOPFGHUZFnaTs+lIzzxsBa+0uTKzFPGEYiX6WgXGIcae8B9BQ6M8r3VETZWHqyHfuG1emTcSCu5g3710jsu9+IMFrHWibIObmeJpEd7q11+IUJkhGqB000+t8w8/4a2lOwG7UGSbBHxBtA= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1682709178399479.99826112072435; Fri, 28 Apr 2023 12:12:58 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1psTWE-0004oq-SB; Fri, 28 Apr 2023 15:12:34 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1psTWD-0004mR-29 for qemu-devel@nongnu.org; Fri, 28 Apr 2023 15:12:33 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1psTW7-0002yS-KW for qemu-devel@nongnu.org; Fri, 28 Apr 2023 15:12:30 -0400 Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-601-TpTYb0ahOEKewAbz68JEaQ-1; Fri, 28 Apr 2023 15:12:23 -0400 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.rdu2.redhat.com [10.11.54.6]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 2CE5229A9D3F; Fri, 28 Apr 2023 19:12:23 +0000 (UTC) Received: from secure.mitica (unknown [10.39.193.238]) by smtp.corp.redhat.com (Postfix) with ESMTP id BF0872166B4F; Fri, 28 Apr 2023 19:12:21 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1682709146; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=RWuMgvR3+lr0+73m6jOfgll5e5994spFbRjX06qZlj4=; b=Y9wIuhCEYRnannToI70Oo8lJbKtmWUbMBGv8KBQ1BnxOC4ZH9+aiOIGsivOSWU8h/GIh8h 6AIFtaZY5xgBBKoaIDz6DFxrsVHvfoHl5n2pPVxhloxkhTHBFegy723QIgtU3PIaDw5/Sy AS0w2ddS6wsbyhvzslKQ3TQksyjZfuw= X-MC-Unique: TpTYb0ahOEKewAbz68JEaQ-1 From: Juan Quintela To: qemu-devel@nongnu.org Cc: Leonardo Bras , Juan Quintela , Thomas Huth , Laurent Vivier , Paolo Bonzini , Peter Xu , Lukas Straub Subject: [PULL 10/21] qtest/migration-test.c: Add postcopy tests with compress enabled Date: Fri, 28 Apr 2023 21:11:52 +0200 Message-Id: <20230428191203.39520-11-quintela@redhat.com> In-Reply-To: <20230428191203.39520-1-quintela@redhat.com> References: <20230428191203.39520-1-quintela@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.1 on 10.11.54.6 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=quintela@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -22 X-Spam_score: -2.3 X-Spam_bar: -- X-Spam_report: (-2.3 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.171, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1682709180553100008 Content-Type: text/plain; charset="utf-8" From: Lukas Straub Add postcopy tests with compress enabled to ensure nothing breaks with the refactoring in the next commits. preempt+compress is blocked, so no test needed for that case. Signed-off-by: Lukas Straub Reviewed-by: Juan Quintela Signed-off-by: Juan Quintela --- tests/qtest/migration-test.c | 83 +++++++++++++++++++++++------------- 1 file changed, 53 insertions(+), 30 deletions(-) diff --git a/tests/qtest/migration-test.c b/tests/qtest/migration-test.c index d28b0d9880..996f5eceb0 100644 --- a/tests/qtest/migration-test.c +++ b/tests/qtest/migration-test.c @@ -1127,6 +1127,36 @@ test_migrate_tls_x509_finish(QTestState *from, #endif /* CONFIG_TASN1 */ #endif /* CONFIG_GNUTLS */ =20 +static void * +test_migrate_compress_start(QTestState *from, + QTestState *to) +{ + migrate_set_parameter_int(from, "compress-level", 1); + migrate_set_parameter_int(from, "compress-threads", 4); + migrate_set_parameter_bool(from, "compress-wait-thread", true); + migrate_set_parameter_int(to, "decompress-threads", 4); + + migrate_set_capability(from, "compress", true); + migrate_set_capability(to, "compress", true); + + return NULL; +} + +static void * +test_migrate_compress_nowait_start(QTestState *from, + QTestState *to) +{ + migrate_set_parameter_int(from, "compress-level", 9); + migrate_set_parameter_int(from, "compress-threads", 1); + migrate_set_parameter_bool(from, "compress-wait-thread", false); + migrate_set_parameter_int(to, "decompress-threads", 1); + + migrate_set_capability(from, "compress", true); + migrate_set_capability(to, "compress", true); + + return NULL; +} + static int migrate_postcopy_prepare(QTestState **from_ptr, QTestState **to_ptr, MigrateCommon *args) @@ -1204,6 +1234,15 @@ static void test_postcopy(void) test_postcopy_common(&args); } =20 +static void test_postcopy_compress(void) +{ + MigrateCommon args =3D { + .start_hook =3D test_migrate_compress_start + }; + + test_postcopy_common(&args); +} + static void test_postcopy_preempt(void) { MigrateCommon args =3D { @@ -1305,6 +1344,15 @@ static void test_postcopy_recovery(void) test_postcopy_recovery_common(&args); } =20 +static void test_postcopy_recovery_compress(void) +{ + MigrateCommon args =3D { + .start_hook =3D test_migrate_compress_start + }; + + test_postcopy_recovery_common(&args); +} + #ifdef CONFIG_GNUTLS static void test_postcopy_recovery_tls_psk(void) { @@ -1338,6 +1386,7 @@ static void test_postcopy_preempt_all(void) =20 test_postcopy_recovery_common(&args); } + #endif =20 static void test_baddest(void) @@ -1559,21 +1608,6 @@ static void test_precopy_unix_xbzrle(void) test_precopy_common(&args); } =20 -static void * -test_migrate_compress_start(QTestState *from, - QTestState *to) -{ - migrate_set_parameter_int(from, "compress-level", 1); - migrate_set_parameter_int(from, "compress-threads", 4); - migrate_set_parameter_bool(from, "compress-wait-thread", true); - migrate_set_parameter_int(to, "decompress-threads", 4); - - migrate_set_capability(from, "compress", true); - migrate_set_capability(to, "compress", true); - - return NULL; -} - static void test_precopy_unix_compress(void) { g_autofree char *uri =3D g_strdup_printf("unix:%s/migsocket", tmpfs); @@ -1591,21 +1625,6 @@ static void test_precopy_unix_compress(void) test_precopy_common(&args); } =20 -static void * -test_migrate_compress_nowait_start(QTestState *from, - QTestState *to) -{ - migrate_set_parameter_int(from, "compress-level", 9); - migrate_set_parameter_int(from, "compress-threads", 1); - migrate_set_parameter_bool(from, "compress-wait-thread", false); - migrate_set_parameter_int(to, "decompress-threads", 1); - - migrate_set_capability(from, "compress", true); - migrate_set_capability(to, "compress", true); - - return NULL; -} - static void test_precopy_unix_compress_nowait(void) { g_autofree char *uri =3D g_strdup_printf("unix:%s/migsocket", tmpfs); @@ -2619,8 +2638,12 @@ int main(int argc, char **argv) =20 if (has_uffd) { qtest_add_func("/migration/postcopy/plain", test_postcopy); + qtest_add_func("/migration/postcopy/compress/plain", + test_postcopy_compress); qtest_add_func("/migration/postcopy/recovery/plain", test_postcopy_recovery); + qtest_add_func("/migration/postcopy/recovery/compress/plain", + test_postcopy_recovery_compress); qtest_add_func("/migration/postcopy/preempt/plain", test_postcopy_= preempt); qtest_add_func("/migration/postcopy/preempt/recovery/plain", test_postcopy_preempt_recovery); --=20 2.40.0 From nobody Fri May 17 02:03:38 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1682709235; cv=none; d=zohomail.com; s=zohoarc; b=Ar7+KptD1V/2IeocnxPowkqo5vVLZl43w+8e2P2GMm7DCL+XpKQHO7X2wxfhp0mQ2avREi9BpGfszDDhPaAtwF9cY+nEY2n/JD/nm5tn8MMfh5eajRL9EI19ZKPJL/9QAY9KzJcpVyz0Oaw4WUakguuSfAQ83X1yxUg0Rr3qQ9E= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1682709235; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=/wIkfEC2nYbrAsqYyql5HAgC3ikXBlVEGcKuXPFR1uc=; b=IZtnF3wjnvU0X0fnBEkJXXjnu+OGwpUjO1SYaqoOOnDh3fCs43v6COb9CSABfThEqdGNlHUGZeAp/7/6Mcakll2lM/a1fw5/SZ4KbHC1IoVEb9aOryP0iZAPcq8bs4w1REUL3OXaPgZlOS87V4yR1VDG29ckB3atAvnBrC/4s4I= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1682709235080142.8066629574946; Fri, 28 Apr 2023 12:13:55 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1psTWG-0004r2-Uh; Fri, 28 Apr 2023 15:12:36 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1psTWF-0004pB-AB for qemu-devel@nongnu.org; Fri, 28 Apr 2023 15:12:35 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1psTWC-0002zP-Eg for qemu-devel@nongnu.org; Fri, 28 Apr 2023 15:12:35 -0400 Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-150-1Kd3-Gf8OOyozC4HSBiwtg-1; Fri, 28 Apr 2023 15:12:26 -0400 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.rdu2.redhat.com [10.11.54.6]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id C08981C06914; Fri, 28 Apr 2023 19:12:24 +0000 (UTC) Received: from secure.mitica (unknown [10.39.193.238]) by smtp.corp.redhat.com (Postfix) with ESMTP id 71A262166B4F; Fri, 28 Apr 2023 19:12:23 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1682709149; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=/wIkfEC2nYbrAsqYyql5HAgC3ikXBlVEGcKuXPFR1uc=; b=cSrpnRno9Vrnmi7akh8fTpgkeRxqFThRlAH4PXrSKgEVI1Jgx8uitL9ZT+EOYAMu77enOv MrjkcETaLwACES8kJVl7NyRNXYQBLLTcw/ct1zxgain7NEnW+HB+VcVBfQ9IiUG/UCeghT qI3uVO22W9JR9lNoXRFzJRTPZWJ/qAs= X-MC-Unique: 1Kd3-Gf8OOyozC4HSBiwtg-1 From: Juan Quintela To: qemu-devel@nongnu.org Cc: Leonardo Bras , Juan Quintela , Thomas Huth , Laurent Vivier , Paolo Bonzini , Peter Xu , Lukas Straub Subject: [PULL 11/21] ram.c: Let the compress threads return a CompressResult enum Date: Fri, 28 Apr 2023 21:11:53 +0200 Message-Id: <20230428191203.39520-12-quintela@redhat.com> In-Reply-To: <20230428191203.39520-1-quintela@redhat.com> References: <20230428191203.39520-1-quintela@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.1 on 10.11.54.6 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=quintela@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -22 X-Spam_score: -2.3 X-Spam_bar: -- X-Spam_report: (-2.3 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.171, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1682709236875100002 Content-Type: text/plain; charset="utf-8" From: Lukas Straub This will be used in the next commits to move save_page_header() out of compress code. Signed-off-by: Lukas Straub Reviewed-by: Juan Quintela Signed-off-by: Juan Quintela --- migration/ram.c | 34 ++++++++++++++++++++++------------ 1 file changed, 22 insertions(+), 12 deletions(-) diff --git a/migration/ram.c b/migration/ram.c index 7d81c4a39e..2b213b923b 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -482,10 +482,17 @@ MigrationOps *migration_ops; =20 CompressionStats compression_counters; =20 +enum CompressResult { + RES_NONE =3D 0, + RES_ZEROPAGE =3D 1, + RES_COMPRESS =3D 2 +}; +typedef enum CompressResult CompressResult; + struct CompressParam { bool done; bool quit; - bool zero_page; + CompressResult result; QEMUFile *file; QemuMutex mutex; QemuCond cond; @@ -527,8 +534,9 @@ static QemuCond decomp_done_cond; =20 static int ram_save_host_page_urgent(PageSearchStatus *pss); =20 -static bool do_compress_ram_page(QEMUFile *f, z_stream *stream, RAMBlock *= block, - ram_addr_t offset, uint8_t *source_buf); +static CompressResult do_compress_ram_page(QEMUFile *f, z_stream *stream, + RAMBlock *block, ram_addr_t off= set, + uint8_t *source_buf); =20 /* NOTE: page is the PFN not real ram_addr_t. */ static void pss_init(PageSearchStatus *pss, RAMBlock *rb, ram_addr_t page) @@ -553,7 +561,7 @@ static void *do_data_compress(void *opaque) CompressParam *param =3D opaque; RAMBlock *block; ram_addr_t offset; - bool zero_page; + CompressResult result; =20 qemu_mutex_lock(¶m->mutex); while (!param->quit) { @@ -563,12 +571,12 @@ static void *do_data_compress(void *opaque) param->block =3D NULL; qemu_mutex_unlock(¶m->mutex); =20 - zero_page =3D do_compress_ram_page(param->file, ¶m->stream, - block, offset, param->originb= uf); + result =3D do_compress_ram_page(param->file, ¶m->stream, + block, offset, param->originbuf); =20 qemu_mutex_lock(&comp_done_lock); param->done =3D true; - param->zero_page =3D zero_page; + param->result =3D result; qemu_cond_signal(&comp_done_cond); qemu_mutex_unlock(&comp_done_lock); =20 @@ -1452,8 +1460,9 @@ static int ram_save_multifd_page(QEMUFile *file, RAMB= lock *block, return 1; } =20 -static bool do_compress_ram_page(QEMUFile *f, z_stream *stream, RAMBlock *= block, - ram_addr_t offset, uint8_t *source_buf) +static CompressResult do_compress_ram_page(QEMUFile *f, z_stream *stream, + RAMBlock *block, ram_addr_t off= set, + uint8_t *source_buf) { RAMState *rs =3D ram_state; PageSearchStatus *pss =3D &rs->pss[RAM_CHANNEL_PRECOPY]; @@ -1461,7 +1470,7 @@ static bool do_compress_ram_page(QEMUFile *f, z_strea= m *stream, RAMBlock *block, int ret; =20 if (save_zero_page_to_file(pss, f, block, offset)) { - return true; + return RES_ZEROPAGE; } =20 save_page_header(pss, f, block, offset | RAM_SAVE_FLAG_COMPRESS_PAGE); @@ -1476,8 +1485,9 @@ static bool do_compress_ram_page(QEMUFile *f, z_strea= m *stream, RAMBlock *block, if (ret < 0) { qemu_file_set_error(migrate_get_current()->to_dst_file, ret); error_report("compressed data failed!"); + return RES_NONE; } - return false; + return RES_COMPRESS; } =20 static void @@ -1485,7 +1495,7 @@ update_compress_thread_counts(const CompressParam *pa= ram, int bytes_xmit) { ram_transferred_add(bytes_xmit); =20 - if (param->zero_page) { + if (param->result =3D=3D RES_ZEROPAGE) { stat64_add(&mig_stats.zero_pages, 1); return; } --=20 2.40.0 From nobody Fri May 17 02:03:38 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1682709179; cv=none; d=zohomail.com; s=zohoarc; b=NtJWDAIowEmKTe+pB2wa4KP4ieunW90grDZkWZGUmPDmJyRyoqcFXMlxHgBjCESGzmqBpbAJqysPZZeRhRI/Q0+0qmgwAmWHoO0RrsmyhdXyPPxj35O/qXmTlR1NDRQl8ad+IR4e5lWR64mPdOdqq0xxbxhYKrmKcfSuGLbQ+Ug= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1682709179; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=SrtWU4eYpPxCG2l34+hRkJBdbAa/0xR54AuRtWlizBw=; b=YrnGxKbH5bWU6QebEygPhP0+NGkUjHmAV8TRO4v/K3H4SlZE0I7NHjQso6LP8JCMogtZmL43YMI8kMuGVO4ZDo0/bX7PBlQk1UtjsqifITk9j0XHrWmX+wS/wpGTjXZpFMk5DRVmw5bsJpboZhX88bwDya3PUzniPLT69ggBVWY= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1682709179415564.8239279437661; Fri, 28 Apr 2023 12:12:59 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1psTWI-0004sZ-Hm; Fri, 28 Apr 2023 15:12:38 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1psTWG-0004qD-Bx for qemu-devel@nongnu.org; Fri, 28 Apr 2023 15:12:36 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1psTWC-0002zb-QO for qemu-devel@nongnu.org; Fri, 28 Apr 2023 15:12:36 -0400 Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-637-yzozoQQMPcmSenasyxcFnQ-1; Fri, 28 Apr 2023 15:12:26 -0400 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.rdu2.redhat.com [10.11.54.6]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 75D8F858F0E; Fri, 28 Apr 2023 19:12:26 +0000 (UTC) Received: from secure.mitica (unknown [10.39.193.238]) by smtp.corp.redhat.com (Postfix) with ESMTP id 2643D2166B4F; Fri, 28 Apr 2023 19:12:24 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1682709150; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=SrtWU4eYpPxCG2l34+hRkJBdbAa/0xR54AuRtWlizBw=; b=a8qJaJPdWXcqSqz96JXseheFFH1A3hbiO944bZNHgkDtRdl1X2ago4ABDpyyAUEVF+P+S7 mgETi4nGUXvzB7nR3UxjTry+Q6yaBB90XY4w13zhN+E/yPcMwTGXW5x0/FL47mcmcOHt/O do9bAb+1B2AfAh1DJvpAtD2iHG9JTAI= X-MC-Unique: yzozoQQMPcmSenasyxcFnQ-1 From: Juan Quintela To: qemu-devel@nongnu.org Cc: Leonardo Bras , Juan Quintela , Thomas Huth , Laurent Vivier , Paolo Bonzini , Peter Xu , Lukas Straub Subject: [PULL 12/21] ram.c: Dont change param->block in the compress thread Date: Fri, 28 Apr 2023 21:11:54 +0200 Message-Id: <20230428191203.39520-13-quintela@redhat.com> In-Reply-To: <20230428191203.39520-1-quintela@redhat.com> References: <20230428191203.39520-1-quintela@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.1 on 10.11.54.6 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=quintela@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -22 X-Spam_score: -2.3 X-Spam_bar: -- X-Spam_report: (-2.3 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.171, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1682709180005100005 Content-Type: text/plain; charset="utf-8" From: Lukas Straub Instead introduce a extra parameter to trigger the compress thread. Now, when the compress thread is done, we know what RAMBlock and offset it did compress. This will be used in the next commits to move save_page_header() out of compress code. Signed-off-by: Lukas Straub Reviewed-by: Juan Quintela Signed-off-by: Juan Quintela --- migration/ram.c | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/migration/ram.c b/migration/ram.c index 2b213b923b..ebcde3dcd4 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -492,6 +492,7 @@ typedef enum CompressResult CompressResult; struct CompressParam { bool done; bool quit; + bool trigger; CompressResult result; QEMUFile *file; QemuMutex mutex; @@ -565,10 +566,10 @@ static void *do_data_compress(void *opaque) =20 qemu_mutex_lock(¶m->mutex); while (!param->quit) { - if (param->block) { + if (param->trigger) { block =3D param->block; offset =3D param->offset; - param->block =3D NULL; + param->trigger =3D false; qemu_mutex_unlock(¶m->mutex); =20 result =3D do_compress_ram_page(param->file, ¶m->stream, @@ -1545,6 +1546,7 @@ static inline void set_compress_params(CompressParam = *param, RAMBlock *block, { param->block =3D block; param->offset =3D offset; + param->trigger =3D true; } =20 static int compress_page_with_multi_thread(RAMBlock *block, ram_addr_t off= set) --=20 2.40.0 From nobody Fri May 17 02:03:38 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1682709200; cv=none; d=zohomail.com; s=zohoarc; b=AnoxjhgWzDiAeEcfmRFByBHJEeyJNAElQisIDYEs9PwkaHqw1FdASPRd+zFXfRdHbdGnxYnZwiT/32265BnHe838snBUPK2s/Mp/jPnKB38JxZqyyQeI1uIoReXxzHd7z7qvHwgx6eoYFG8XgxLJtP07c9G+yTwVO66yl+SvGmw= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1682709200; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=fxL2HakFEpP7ql8iErCK7aMLa224UnLvLoHB0puQbgg=; b=gjBvlC9pl9zxlBUi55Ao3fnmY0g/bNmeZehd3nlhNXEBkho5lJLz7XScUiJbZXVI5/N+Z1EXaiZK+cxHoOMkN35hLH/703HIfQHCurIIUuPBuvMkApXa2bE6almdmz9cTFUwlvsFKgfUYNOdQtxWI0sQ17/fqWFBLM3zNgSS8+M= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1682709200825309.817086455918; Fri, 28 Apr 2023 12:13:20 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1psTWK-0004tv-CS; Fri, 28 Apr 2023 15:12:40 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1psTWJ-0004t4-3h for qemu-devel@nongnu.org; Fri, 28 Apr 2023 15:12:39 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1psTWH-00031X-JC for qemu-devel@nongnu.org; Fri, 28 Apr 2023 15:12:38 -0400 Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-151-JzfH0qfLMX6qP8gl-SIzBQ-1; Fri, 28 Apr 2023 15:12:30 -0400 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.rdu2.redhat.com [10.11.54.6]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 23122858F09; Fri, 28 Apr 2023 19:12:28 +0000 (UTC) Received: from secure.mitica (unknown [10.39.193.238]) by smtp.corp.redhat.com (Postfix) with ESMTP id B76A12166B4F; Fri, 28 Apr 2023 19:12:26 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1682709156; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=fxL2HakFEpP7ql8iErCK7aMLa224UnLvLoHB0puQbgg=; b=ZZQEg59xvFinxDDCi9YZgfEfW4Awbcg4qCKj3RaRzPz2CKnMtf57jNzrF3xu00YIA/UcH1 SprR6Xw58dQfe/aaiWF0PLG5uxGaZfROSnSaov0mJo+fu5pk0+4YuNkhYR5EV6FbrkrAYX lzYPN7WzYmYoH0ArpUfRLYheX1NA4W0= X-MC-Unique: JzfH0qfLMX6qP8gl-SIzBQ-1 From: Juan Quintela To: qemu-devel@nongnu.org Cc: Leonardo Bras , Juan Quintela , Thomas Huth , Laurent Vivier , Paolo Bonzini , Peter Xu , Lukas Straub Subject: [PULL 13/21] ram.c: Reset result after sending queued data Date: Fri, 28 Apr 2023 21:11:55 +0200 Message-Id: <20230428191203.39520-14-quintela@redhat.com> In-Reply-To: <20230428191203.39520-1-quintela@redhat.com> References: <20230428191203.39520-1-quintela@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.1 on 10.11.54.6 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=quintela@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -22 X-Spam_score: -2.3 X-Spam_bar: -- X-Spam_report: (-2.3 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.171, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1682709201513100001 Content-Type: text/plain; charset="utf-8" From: Lukas Straub And take the param->mutex lock for the whole section to ensure thread-safety. Now, it is explicitly clear if there is no queued data to send. Before, this was handled by param->file stream being empty and thus qemu_put_qemu_file() not sending anything. This will be used in the next commits to move save_page_header() out of compress code. Signed-off-by: Lukas Straub Reviewed-by: Juan Quintela Signed-off-by: Juan Quintela --- migration/ram.c | 32 ++++++++++++++++++++++---------- 1 file changed, 22 insertions(+), 10 deletions(-) diff --git a/migration/ram.c b/migration/ram.c index ebcde3dcd4..b0b11f37fb 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -1508,6 +1508,13 @@ update_compress_thread_counts(const CompressParam *p= aram, int bytes_xmit) =20 static bool save_page_use_compression(RAMState *rs); =20 +static inline void compress_reset_result(CompressParam *param) +{ + param->result =3D RES_NONE; + param->block =3D NULL; + param->offset =3D 0; +} + static void flush_compressed_data(RAMState *rs) { MigrationState *ms =3D migrate_get_current(); @@ -1529,13 +1536,16 @@ static void flush_compressed_data(RAMState *rs) for (idx =3D 0; idx < thread_count; idx++) { qemu_mutex_lock(&comp_param[idx].mutex); if (!comp_param[idx].quit) { - len =3D qemu_put_qemu_file(ms->to_dst_file, comp_param[idx].fi= le); + CompressParam *param =3D &comp_param[idx]; + len =3D qemu_put_qemu_file(ms->to_dst_file, param->file); + compress_reset_result(param); + /* * it's safe to fetch zero_page without holding comp_done_lock * as there is no further request submitted to the thread, * i.e, the thread should be waiting for a request at this poi= nt. */ - update_compress_thread_counts(&comp_param[idx], len); + update_compress_thread_counts(param, len); } qemu_mutex_unlock(&comp_param[idx].mutex); } @@ -1560,15 +1570,17 @@ static int compress_page_with_multi_thread(RAMBlock= *block, ram_addr_t offset) retry: for (idx =3D 0; idx < thread_count; idx++) { if (comp_param[idx].done) { - comp_param[idx].done =3D false; - bytes_xmit =3D qemu_put_qemu_file(ms->to_dst_file, - comp_param[idx].file); - qemu_mutex_lock(&comp_param[idx].mutex); - set_compress_params(&comp_param[idx], block, offset); - qemu_cond_signal(&comp_param[idx].cond); - qemu_mutex_unlock(&comp_param[idx].mutex); + CompressParam *param =3D &comp_param[idx]; + qemu_mutex_lock(¶m->mutex); + param->done =3D false; + bytes_xmit =3D qemu_put_qemu_file(ms->to_dst_file, param->file= ); + compress_reset_result(param); + set_compress_params(param, block, offset); + + update_compress_thread_counts(param, bytes_xmit); + qemu_cond_signal(¶m->cond); + qemu_mutex_unlock(¶m->mutex); pages =3D 1; - update_compress_thread_counts(&comp_param[idx], bytes_xmit); break; } } --=20 2.40.0 From nobody Fri May 17 02:03:38 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1682709269; cv=none; d=zohomail.com; s=zohoarc; b=Wkgxopsehw6tGihv7AKPUDagJ4/ofEViMamA5owlIcHC5t7NFy71ndUbGWTv62a6PSHS+FMcFBXml6jWgX5Ka2OCiZGz2CDzEwCuiljVkyNxP/Cda6IYMPN2s/6r4ra3Pxx68inltC4KIk/+Jj8jo0WVz9crUdETRcpGt2mhoEY= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1682709269; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=B4sv46wwHMZHoYs6QFZax8TYeOmxvB56IfSEwn1UIhU=; b=ZlmEEM8JpqN6QI6vqY/ghxv4z+14Y9aOtpQBPpxkC//hMeO2H9j6wh48yo8GvCbvlOPGYogfgSISX0k6B13CH5vhM4ueUbkHVEwiAk+/uIBQUxZLFMO7O6cmvJuUp00HrXsiCWhs5FuywgOxmnNkuJbJ4t+Okzu8vpxK2F34q0M= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1682709269430197.23563683975067; Fri, 28 Apr 2023 12:14:29 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1psTWJ-0004t1-2z; Fri, 28 Apr 2023 15:12:39 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1psTWG-0004qL-H1 for qemu-devel@nongnu.org; Fri, 28 Apr 2023 15:12:36 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1psTWC-0002zv-QN for qemu-devel@nongnu.org; Fri, 28 Apr 2023 15:12:36 -0400 Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-652-ysCqC1yMNy6Yg9U5I8XgDw-1; Fri, 28 Apr 2023 15:12:30 -0400 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.rdu2.redhat.com [10.11.54.6]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id B700D185A78B; Fri, 28 Apr 2023 19:12:29 +0000 (UTC) Received: from secure.mitica (unknown [10.39.193.238]) by smtp.corp.redhat.com (Postfix) with ESMTP id 649C12166B5E; Fri, 28 Apr 2023 19:12:28 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1682709151; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=B4sv46wwHMZHoYs6QFZax8TYeOmxvB56IfSEwn1UIhU=; b=jHwHv05LBSM7pOYGhmWft8c034a/88WSTyC5hR4fRRHSUVvRltnpmPLf0VP4WxRwMewfFP fGdArwqmoBaHoFks79rtL7e5DLqQf2SHPtbseyOOiGALZmCbbSd1hhDdev4EIq1Nyn/1Oh Ai28YU7s9S+SaljtwZ4nVdiqtvHXKVY= X-MC-Unique: ysCqC1yMNy6Yg9U5I8XgDw-1 From: Juan Quintela To: qemu-devel@nongnu.org Cc: Leonardo Bras , Juan Quintela , Thomas Huth , Laurent Vivier , Paolo Bonzini , Peter Xu , Lukas Straub Subject: [PULL 14/21] ram.c: Do not call save_page_header() from compress threads Date: Fri, 28 Apr 2023 21:11:56 +0200 Message-Id: <20230428191203.39520-15-quintela@redhat.com> In-Reply-To: <20230428191203.39520-1-quintela@redhat.com> References: <20230428191203.39520-1-quintela@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.1 on 10.11.54.6 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=quintela@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -22 X-Spam_score: -2.3 X-Spam_bar: -- X-Spam_report: (-2.3 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.171, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1682709270516100005 Content-Type: text/plain; charset="utf-8" From: Lukas Straub save_page_header() accesses several global variables, so calling it from multiple threads is pretty ugly. Instead, call save_page_header() before writing out the compressed data from the compress buffer to the migration stream. This also makes the core compress code more independend from ram.c. Signed-off-by: Lukas Straub Reviewed-by: Juan Quintela Signed-off-by: Juan Quintela --- migration/ram.c | 44 +++++++++++++++++++++++++++++++++++--------- 1 file changed, 35 insertions(+), 9 deletions(-) diff --git a/migration/ram.c b/migration/ram.c index b0b11f37fb..53aba37be9 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -1465,17 +1465,13 @@ static CompressResult do_compress_ram_page(QEMUFile= *f, z_stream *stream, RAMBlock *block, ram_addr_t off= set, uint8_t *source_buf) { - RAMState *rs =3D ram_state; - PageSearchStatus *pss =3D &rs->pss[RAM_CHANNEL_PRECOPY]; uint8_t *p =3D block->host + offset; int ret; =20 - if (save_zero_page_to_file(pss, f, block, offset)) { + if (buffer_is_zero(p, TARGET_PAGE_SIZE)) { return RES_ZEROPAGE; } =20 - save_page_header(pss, f, block, offset | RAM_SAVE_FLAG_COMPRESS_PAGE); - /* * copy it to a internal buffer to avoid it being modified by VM * so that we can catch up the error during compression and @@ -1515,9 +1511,40 @@ static inline void compress_reset_result(CompressPar= am *param) param->offset =3D 0; } =20 +static int send_queued_data(CompressParam *param) +{ + PageSearchStatus *pss =3D &ram_state->pss[RAM_CHANNEL_PRECOPY]; + MigrationState *ms =3D migrate_get_current(); + QEMUFile *file =3D ms->to_dst_file; + int len =3D 0; + + RAMBlock *block =3D param->block; + ram_addr_t offset =3D param->offset; + + if (param->result =3D=3D RES_NONE) { + return 0; + } + + assert(block =3D=3D pss->last_sent_block); + + if (param->result =3D=3D RES_ZEROPAGE) { + len +=3D save_page_header(pss, file, block, offset | RAM_SAVE_FLAG= _ZERO); + qemu_put_byte(file, 0); + len +=3D 1; + ram_release_page(block->idstr, offset); + } else if (param->result =3D=3D RES_COMPRESS) { + len +=3D save_page_header(pss, file, block, + offset | RAM_SAVE_FLAG_COMPRESS_PAGE); + len +=3D qemu_put_qemu_file(file, param->file); + } else { + abort(); + } + + return len; +} + static void flush_compressed_data(RAMState *rs) { - MigrationState *ms =3D migrate_get_current(); int idx, len, thread_count; =20 if (!save_page_use_compression(rs)) { @@ -1537,7 +1564,7 @@ static void flush_compressed_data(RAMState *rs) qemu_mutex_lock(&comp_param[idx].mutex); if (!comp_param[idx].quit) { CompressParam *param =3D &comp_param[idx]; - len =3D qemu_put_qemu_file(ms->to_dst_file, param->file); + len =3D send_queued_data(param); compress_reset_result(param); =20 /* @@ -1563,7 +1590,6 @@ static int compress_page_with_multi_thread(RAMBlock *= block, ram_addr_t offset) { int idx, thread_count, bytes_xmit =3D -1, pages =3D -1; bool wait =3D migrate_compress_wait_thread(); - MigrationState *ms =3D migrate_get_current(); =20 thread_count =3D migrate_compress_threads(); qemu_mutex_lock(&comp_done_lock); @@ -1573,7 +1599,7 @@ retry: CompressParam *param =3D &comp_param[idx]; qemu_mutex_lock(¶m->mutex); param->done =3D false; - bytes_xmit =3D qemu_put_qemu_file(ms->to_dst_file, param->file= ); + bytes_xmit =3D send_queued_data(param); compress_reset_result(param); set_compress_params(param, block, offset); =20 --=20 2.40.0 From nobody Fri May 17 02:03:38 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1682709238; cv=none; d=zohomail.com; s=zohoarc; b=arWJyVcGS7I790DJMtxQRVLrAjVe9vH0AzbiMUSsnCtFofdVV7HdFIduocUFqltcm9QWnefawGHc5qEsIYfvRvpkrNQJpx2fTWttHvWsGeDp2NDR4S2nKs9QuV/BwFNk4gdZP5sJoyZpW3cV20cnNHwT+aV8ej48oBskrpxWquE= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1682709238; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=su5i0UfNaL2XCe4Ha2zLPv/vFQSyMEgy7eGwIEg2rr4=; b=V4th+r6yZvBA3vngT7P/D//jxxzZuzoR1W+2zXlGTFUUeTXytn7vs9Ivx9pSxbb4OFxJQAmOJNwaYDaJ5g98HMeXgU37BQV6DN9Ycd+DMp6qhaEUmQn4vWhVxsdWIPXTPVWP9wnHTUvMv95yYqVWMySH++8SLjLgDmdy4jqW79w= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1682709238688751.3132951991256; Fri, 28 Apr 2023 12:13:58 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1psTWL-0004uY-Fw; Fri, 28 Apr 2023 15:12:41 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1psTWH-0004ra-Fe for qemu-devel@nongnu.org; Fri, 28 Apr 2023 15:12:37 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1psTWF-000311-QR for qemu-devel@nongnu.org; Fri, 28 Apr 2023 15:12:37 -0400 Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-646-0G-Xu9BvMkqvYQvkI7zwPQ-1; Fri, 28 Apr 2023 15:12:31 -0400 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.rdu2.redhat.com [10.11.54.6]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 6DF953C025C0; Fri, 28 Apr 2023 19:12:31 +0000 (UTC) Received: from secure.mitica (unknown [10.39.193.238]) by smtp.corp.redhat.com (Postfix) with ESMTP id 060B02166B4F; Fri, 28 Apr 2023 19:12:29 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1682709155; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=su5i0UfNaL2XCe4Ha2zLPv/vFQSyMEgy7eGwIEg2rr4=; b=P0kJ47VIjz2k6vkNO+sN0q9DRjk1+E12RrZge0826x8obatk1y4uy7/PTrpNuKX0zLWU83 s68SCHZCC+by2lCFmuFT8X7WzMuvlEQEt6bK1u0Qy2u0Z271QoM+OUYuAd7NLkSgsE/Cky I+9w5modE3LiK0UgFWLt/2LouHFjWY0= X-MC-Unique: 0G-Xu9BvMkqvYQvkI7zwPQ-1 From: Juan Quintela To: qemu-devel@nongnu.org Cc: Leonardo Bras , Juan Quintela , Thomas Huth , Laurent Vivier , Paolo Bonzini , Peter Xu , Lukas Straub Subject: [PULL 15/21] ram.c: Call update_compress_thread_counts from compress_send_queued_data Date: Fri, 28 Apr 2023 21:11:57 +0200 Message-Id: <20230428191203.39520-16-quintela@redhat.com> In-Reply-To: <20230428191203.39520-1-quintela@redhat.com> References: <20230428191203.39520-1-quintela@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.1 on 10.11.54.6 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=quintela@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -22 X-Spam_score: -2.3 X-Spam_bar: -- X-Spam_report: (-2.3 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.171, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1682709239661100003 Content-Type: text/plain; charset="utf-8" From: Lukas Straub This makes the core compress code more independend from ram.c. Signed-off-by: Lukas Straub Reviewed-by: Juan Quintela Signed-off-by: Juan Quintela --- migration/ram.c | 18 ++++++------------ 1 file changed, 6 insertions(+), 12 deletions(-) diff --git a/migration/ram.c b/migration/ram.c index 53aba37be9..f17f6581ff 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -1540,12 +1540,14 @@ static int send_queued_data(CompressParam *param) abort(); } =20 + update_compress_thread_counts(param, len); + return len; } =20 static void flush_compressed_data(RAMState *rs) { - int idx, len, thread_count; + int idx, thread_count; =20 if (!save_page_use_compression(rs)) { return; @@ -1564,15 +1566,8 @@ static void flush_compressed_data(RAMState *rs) qemu_mutex_lock(&comp_param[idx].mutex); if (!comp_param[idx].quit) { CompressParam *param =3D &comp_param[idx]; - len =3D send_queued_data(param); + send_queued_data(param); compress_reset_result(param); - - /* - * it's safe to fetch zero_page without holding comp_done_lock - * as there is no further request submitted to the thread, - * i.e, the thread should be waiting for a request at this poi= nt. - */ - update_compress_thread_counts(param, len); } qemu_mutex_unlock(&comp_param[idx].mutex); } @@ -1588,7 +1583,7 @@ static inline void set_compress_params(CompressParam = *param, RAMBlock *block, =20 static int compress_page_with_multi_thread(RAMBlock *block, ram_addr_t off= set) { - int idx, thread_count, bytes_xmit =3D -1, pages =3D -1; + int idx, thread_count, pages =3D -1; bool wait =3D migrate_compress_wait_thread(); =20 thread_count =3D migrate_compress_threads(); @@ -1599,11 +1594,10 @@ retry: CompressParam *param =3D &comp_param[idx]; qemu_mutex_lock(¶m->mutex); param->done =3D false; - bytes_xmit =3D send_queued_data(param); + send_queued_data(param); compress_reset_result(param); set_compress_params(param, block, offset); =20 - update_compress_thread_counts(param, bytes_xmit); qemu_cond_signal(¶m->cond); qemu_mutex_unlock(¶m->mutex); pages =3D 1; --=20 2.40.0 From nobody Fri May 17 02:03:38 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1682709410; cv=none; d=zohomail.com; s=zohoarc; b=UDY3vwrq50aXa+ZUAQJnhkdU5Lov8buYfndWTsGOnDuuUf5hi1I6L+OdMCYPZK/iY/9OnMmG7fnqGkfxW9CQ2TL8bKDQpHe36L0zZGF612oI1j0jcAjteXDLogwLoxaKsFVEgkPwMp6s8gLBzV102uyC3pv9gFIYpp/EQYv0KNI= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1682709410; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=wQ8no8yXnvZofbPeEvNkFyfsWU+Bd+oHB7XDIuJq5kU=; b=ZFnrJJpD4Wf2lt4cRyXpqF9Lp/Uewspx4pD8ZKWLl8D2ZtVJ1Xj94SOUWj+er4nPUkbULrs2B6ko3skWTiTVvehTrDKZlmTPZlkNXeBcXPU2OlMFPwUQYcLPqP7FhIwmIz/tAyu1Jt1VuS2EiT+CuduonBG9/aIl9Wdo94b3SAY= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1682709410784750.6868056305526; Fri, 28 Apr 2023 12:16:50 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1psTWM-0004uw-4g; Fri, 28 Apr 2023 15:12:42 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1psTWI-0004rG-2q for qemu-devel@nongnu.org; Fri, 28 Apr 2023 15:12:38 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1psTWF-00030w-Ia for qemu-devel@nongnu.org; Fri, 28 Apr 2023 15:12:36 -0400 Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-636-Ru6IZWIzOVi9761Gslufzg-1; Fri, 28 Apr 2023 15:12:33 -0400 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.rdu2.redhat.com [10.11.54.6]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 252631C06914; Fri, 28 Apr 2023 19:12:33 +0000 (UTC) Received: from secure.mitica (unknown [10.39.193.238]) by smtp.corp.redhat.com (Postfix) with ESMTP id B61C72166B4F; Fri, 28 Apr 2023 19:12:31 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1682709155; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=wQ8no8yXnvZofbPeEvNkFyfsWU+Bd+oHB7XDIuJq5kU=; b=jPc4nTOdevs56bkahCyJoEjurERiuxH5LqtAp/TNbwF0DPaPkiOaH1LSnvc+3fa7fllQ4E IemFxSVL+gZfaa53iNpGgHL/P91ZGj8yId8KG9s0Rjim+2zQ8MFGQADsK21VGf52RhDJmM SwzCDcunpgVN2XEgb+k0wmr9e6H4D2U= X-MC-Unique: Ru6IZWIzOVi9761Gslufzg-1 From: Juan Quintela To: qemu-devel@nongnu.org Cc: Leonardo Bras , Juan Quintela , Thomas Huth , Laurent Vivier , Paolo Bonzini , Peter Xu , Lukas Straub Subject: [PULL 16/21] ram.c: Remove last ram.c dependency from the core compress code Date: Fri, 28 Apr 2023 21:11:58 +0200 Message-Id: <20230428191203.39520-17-quintela@redhat.com> In-Reply-To: <20230428191203.39520-1-quintela@redhat.com> References: <20230428191203.39520-1-quintela@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.1 on 10.11.54.6 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=quintela@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -22 X-Spam_score: -2.3 X-Spam_bar: -- X-Spam_report: (-2.3 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.171, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1682709411824100001 Content-Type: text/plain; charset="utf-8" From: Lukas Straub Make compression interfaces take send_queued_data() as an argument. Remove save_page_use_compression() from flush_compressed_data(). This removes the last ram.c dependency from the core compress code. Signed-off-by: Lukas Straub Reviewed-by: Juan Quintela Signed-off-by: Juan Quintela --- migration/ram.c | 27 +++++++++++++++++---------- 1 file changed, 17 insertions(+), 10 deletions(-) diff --git a/migration/ram.c b/migration/ram.c index f17f6581ff..3671912b58 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -1545,13 +1545,10 @@ static int send_queued_data(CompressParam *param) return len; } =20 -static void flush_compressed_data(RAMState *rs) +static void flush_compressed_data(int (send_queued_data(CompressParam *))) { int idx, thread_count; =20 - if (!save_page_use_compression(rs)) { - return; - } thread_count =3D migrate_compress_threads(); =20 qemu_mutex_lock(&comp_done_lock); @@ -1573,6 +1570,15 @@ static void flush_compressed_data(RAMState *rs) } } =20 +static void ram_flush_compressed_data(RAMState *rs) +{ + if (!save_page_use_compression(rs)) { + return; + } + + flush_compressed_data(send_queued_data); +} + static inline void set_compress_params(CompressParam *param, RAMBlock *blo= ck, ram_addr_t offset) { @@ -1581,7 +1587,8 @@ static inline void set_compress_params(CompressParam = *param, RAMBlock *block, param->trigger =3D true; } =20 -static int compress_page_with_multi_thread(RAMBlock *block, ram_addr_t off= set) +static int compress_page_with_multi_thread(RAMBlock *block, ram_addr_t off= set, + int (send_queued_data(CompressParam *))) { int idx, thread_count, pages =3D -1; bool wait =3D migrate_compress_wait_thread(); @@ -1672,7 +1679,7 @@ static int find_dirty_block(RAMState *rs, PageSearchS= tatus *pss) * Also If xbzrle is on, stop using the data compression at th= is * point. In theory, xbzrle can do better than compression. */ - flush_compressed_data(rs); + ram_flush_compressed_data(rs); =20 /* Hit the end of the list */ pss->block =3D QLIST_FIRST_RCU(&ram_list.blocks); @@ -2362,11 +2369,11 @@ static bool save_compress_page(RAMState *rs, PageSe= archStatus *pss, * much CPU resource. */ if (block !=3D pss->last_sent_block) { - flush_compressed_data(rs); + ram_flush_compressed_data(rs); return false; } =20 - if (compress_page_with_multi_thread(block, offset) > 0) { + if (compress_page_with_multi_thread(block, offset, send_queued_data) >= 0) { return true; } =20 @@ -3412,7 +3419,7 @@ static int ram_save_iterate(QEMUFile *f, void *opaque) * page is sent in one chunk. */ if (migrate_postcopy_ram()) { - flush_compressed_data(rs); + ram_flush_compressed_data(rs); } =20 /* @@ -3507,7 +3514,7 @@ static int ram_save_complete(QEMUFile *f, void *opaqu= e) } qemu_mutex_unlock(&rs->bitmap_mutex); =20 - flush_compressed_data(rs); + ram_flush_compressed_data(rs); ram_control_after_iterate(f, RAM_CONTROL_FINISH); } =20 --=20 2.40.0 From nobody Fri May 17 02:03:38 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1682709453; cv=none; d=zohomail.com; s=zohoarc; b=cTAu9FeADmtVyNZjD2yU3BXpr75Ge/D6GQmMeReT0UVcqpaPP1hxw9+uiVmbvNFbL3LQIe32VO9+53MtpAv6dWf6/y9CRDdvdYlwH/SZTSSTc9zXzOIldc8pN6dBn1LTinwS8kW6P/FTcF4LgcObcS933Kcx1Ozp8wmH5z+J4yY= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1682709453; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=j7m6xS6BNfkAE4RlJClxgal6tXUEhKbApJckK0teoSc=; b=VUElxFBYzSixTIfyFA82eVYs8mDgCNldbnas5JMxY1yG1W1kDqsk2ZVs9YuvE/tEjcG6+YxRXz1tJCltMtwcsG/f5r1FD+KrHi0Bj1kOc4ddbzV+KjSyIcTDJvsd0u03VkEOtBE6ErQDRnWEcbtwPiadGANG1w4AxEgQL2eOXH0= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1682709453874793.5444143092213; Fri, 28 Apr 2023 12:17:33 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1psTWU-0004yT-4F; Fri, 28 Apr 2023 15:12:50 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1psTWS-0004xb-9L for qemu-devel@nongnu.org; Fri, 28 Apr 2023 15:12:48 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1psTWP-00032J-Hk for qemu-devel@nongnu.org; Fri, 28 Apr 2023 15:12:48 -0400 Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-656-d_t1sN5mP8ymEoMXhtQ9Cg-1; Fri, 28 Apr 2023 15:12:35 -0400 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.rdu2.redhat.com [10.11.54.6]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id DBF9D8314E8; Fri, 28 Apr 2023 19:12:34 +0000 (UTC) Received: from secure.mitica (unknown [10.39.193.238]) by smtp.corp.redhat.com (Postfix) with ESMTP id 6B4162166B4F; Fri, 28 Apr 2023 19:12:33 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1682709164; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=j7m6xS6BNfkAE4RlJClxgal6tXUEhKbApJckK0teoSc=; b=FyEYVlKIYmhtIxd97f+6BP34z4S4YALAEqE0efhDEX5bGQUPxQ/Pep8t+x6wpaWLz9fse0 CAQWUhLFVigaSaHRikUorm6JuS9WCrYBn/E3LSEXZTT5iF8RDGPsUstDT/8LfSeURVPr29 gQx+S57NWra0amk1MIX8QZBdKI1mOCU= X-MC-Unique: d_t1sN5mP8ymEoMXhtQ9Cg-1 From: Juan Quintela To: qemu-devel@nongnu.org Cc: Leonardo Bras , Juan Quintela , Thomas Huth , Laurent Vivier , Paolo Bonzini , Peter Xu , Lukas Straub Subject: [PULL 17/21] ram.c: Move core compression code into its own file Date: Fri, 28 Apr 2023 21:11:59 +0200 Message-Id: <20230428191203.39520-18-quintela@redhat.com> In-Reply-To: <20230428191203.39520-1-quintela@redhat.com> References: <20230428191203.39520-1-quintela@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.1 on 10.11.54.6 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=quintela@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -22 X-Spam_score: -2.3 X-Spam_bar: -- X-Spam_report: (-2.3 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.171, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1682709455941100003 Content-Type: text/plain; charset="utf-8" From: Lukas Straub No functional changes intended. Signed-off-by: Lukas Straub Reviewed-by: Juan Quintela Signed-off-by: Juan Quintela --- migration/meson.build | 5 +- migration/ram-compress.c | 274 +++++++++++++++++++++++++++++++++++++++ migration/ram-compress.h | 65 ++++++++++ migration/ram.c | 262 +------------------------------------ 4 files changed, 344 insertions(+), 262 deletions(-) create mode 100644 migration/ram-compress.c create mode 100644 migration/ram-compress.h diff --git a/migration/meson.build b/migration/meson.build index da1897fadf..2090af8e85 100644 --- a/migration/meson.build +++ b/migration/meson.build @@ -38,4 +38,7 @@ endif softmmu_ss.add(when: zstd, if_true: files('multifd-zstd.c')) =20 specific_ss.add(when: 'CONFIG_SOFTMMU', - if_true: files('dirtyrate.c', 'ram.c', 'target.c')) + if_true: files('dirtyrate.c', + 'ram.c', + 'ram-compress.c', + 'target.c')) diff --git a/migration/ram-compress.c b/migration/ram-compress.c new file mode 100644 index 0000000000..d9bc67d075 --- /dev/null +++ b/migration/ram-compress.c @@ -0,0 +1,274 @@ +/* + * QEMU System Emulator + * + * Copyright (c) 2003-2008 Fabrice Bellard + * Copyright (c) 2011-2015 Red Hat Inc + * + * Authors: + * Juan Quintela + * + * Permission is hereby granted, free of charge, to any person obtaining a= copy + * of this software and associated documentation files (the "Software"), t= o deal + * in the Software without restriction, including without limitation the r= ights + * to use, copy, modify, merge, publish, distribute, sublicense, and/or se= ll + * copies of the Software, and to permit persons to whom the Software is + * furnished to do so, subject to the following conditions: + * + * The above copyright notice and this permission notice shall be included= in + * all copies or substantial portions of the Software. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS= OR + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL + * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OT= HER + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING= FROM, + * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS = IN + * THE SOFTWARE. + */ + +#include "qemu/osdep.h" +#include "qemu/cutils.h" + +#include "ram-compress.h" + +#include "qemu/error-report.h" +#include "migration.h" +#include "options.h" +#include "io/channel-null.h" +#include "exec/ram_addr.h" + +CompressionStats compression_counters; + +static CompressParam *comp_param; +static QemuThread *compress_threads; +/* comp_done_cond is used to wake up the migration thread when + * one of the compression threads has finished the compression. + * comp_done_lock is used to co-work with comp_done_cond. + */ +static QemuMutex comp_done_lock; +static QemuCond comp_done_cond; + +static CompressResult do_compress_ram_page(QEMUFile *f, z_stream *stream, + RAMBlock *block, ram_addr_t off= set, + uint8_t *source_buf); + +static void *do_data_compress(void *opaque) +{ + CompressParam *param =3D opaque; + RAMBlock *block; + ram_addr_t offset; + CompressResult result; + + qemu_mutex_lock(¶m->mutex); + while (!param->quit) { + if (param->trigger) { + block =3D param->block; + offset =3D param->offset; + param->trigger =3D false; + qemu_mutex_unlock(¶m->mutex); + + result =3D do_compress_ram_page(param->file, ¶m->stream, + block, offset, param->originbuf); + + qemu_mutex_lock(&comp_done_lock); + param->done =3D true; + param->result =3D result; + qemu_cond_signal(&comp_done_cond); + qemu_mutex_unlock(&comp_done_lock); + + qemu_mutex_lock(¶m->mutex); + } else { + qemu_cond_wait(¶m->cond, ¶m->mutex); + } + } + qemu_mutex_unlock(¶m->mutex); + + return NULL; +} + +void compress_threads_save_cleanup(void) +{ + int i, thread_count; + + if (!migrate_compress() || !comp_param) { + return; + } + + thread_count =3D migrate_compress_threads(); + for (i =3D 0; i < thread_count; i++) { + /* + * we use it as a indicator which shows if the thread is + * properly init'd or not + */ + if (!comp_param[i].file) { + break; + } + + qemu_mutex_lock(&comp_param[i].mutex); + comp_param[i].quit =3D true; + qemu_cond_signal(&comp_param[i].cond); + qemu_mutex_unlock(&comp_param[i].mutex); + + qemu_thread_join(compress_threads + i); + qemu_mutex_destroy(&comp_param[i].mutex); + qemu_cond_destroy(&comp_param[i].cond); + deflateEnd(&comp_param[i].stream); + g_free(comp_param[i].originbuf); + qemu_fclose(comp_param[i].file); + comp_param[i].file =3D NULL; + } + qemu_mutex_destroy(&comp_done_lock); + qemu_cond_destroy(&comp_done_cond); + g_free(compress_threads); + g_free(comp_param); + compress_threads =3D NULL; + comp_param =3D NULL; +} + +int compress_threads_save_setup(void) +{ + int i, thread_count; + + if (!migrate_compress()) { + return 0; + } + thread_count =3D migrate_compress_threads(); + compress_threads =3D g_new0(QemuThread, thread_count); + comp_param =3D g_new0(CompressParam, thread_count); + qemu_cond_init(&comp_done_cond); + qemu_mutex_init(&comp_done_lock); + for (i =3D 0; i < thread_count; i++) { + comp_param[i].originbuf =3D g_try_malloc(TARGET_PAGE_SIZE); + if (!comp_param[i].originbuf) { + goto exit; + } + + if (deflateInit(&comp_param[i].stream, + migrate_compress_level()) !=3D Z_OK) { + g_free(comp_param[i].originbuf); + goto exit; + } + + /* comp_param[i].file is just used as a dummy buffer to save data, + * set its ops to empty. + */ + comp_param[i].file =3D qemu_file_new_output( + QIO_CHANNEL(qio_channel_null_new())); + comp_param[i].done =3D true; + comp_param[i].quit =3D false; + qemu_mutex_init(&comp_param[i].mutex); + qemu_cond_init(&comp_param[i].cond); + qemu_thread_create(compress_threads + i, "compress", + do_data_compress, comp_param + i, + QEMU_THREAD_JOINABLE); + } + return 0; + +exit: + compress_threads_save_cleanup(); + return -1; +} + +static CompressResult do_compress_ram_page(QEMUFile *f, z_stream *stream, + RAMBlock *block, ram_addr_t off= set, + uint8_t *source_buf) +{ + uint8_t *p =3D block->host + offset; + int ret; + + if (buffer_is_zero(p, TARGET_PAGE_SIZE)) { + return RES_ZEROPAGE; + } + + /* + * copy it to a internal buffer to avoid it being modified by VM + * so that we can catch up the error during compression and + * decompression + */ + memcpy(source_buf, p, TARGET_PAGE_SIZE); + ret =3D qemu_put_compression_data(f, stream, source_buf, TARGET_PAGE_S= IZE); + if (ret < 0) { + qemu_file_set_error(migrate_get_current()->to_dst_file, ret); + error_report("compressed data failed!"); + return RES_NONE; + } + return RES_COMPRESS; +} + +static inline void compress_reset_result(CompressParam *param) +{ + param->result =3D RES_NONE; + param->block =3D NULL; + param->offset =3D 0; +} + +void flush_compressed_data(int (send_queued_data(CompressParam *))) +{ + int idx, thread_count; + + thread_count =3D migrate_compress_threads(); + + qemu_mutex_lock(&comp_done_lock); + for (idx =3D 0; idx < thread_count; idx++) { + while (!comp_param[idx].done) { + qemu_cond_wait(&comp_done_cond, &comp_done_lock); + } + } + qemu_mutex_unlock(&comp_done_lock); + + for (idx =3D 0; idx < thread_count; idx++) { + qemu_mutex_lock(&comp_param[idx].mutex); + if (!comp_param[idx].quit) { + CompressParam *param =3D &comp_param[idx]; + send_queued_data(param); + compress_reset_result(param); + } + qemu_mutex_unlock(&comp_param[idx].mutex); + } +} + +static inline void set_compress_params(CompressParam *param, RAMBlock *blo= ck, + ram_addr_t offset) +{ + param->block =3D block; + param->offset =3D offset; + param->trigger =3D true; +} + +int compress_page_with_multi_thread(RAMBlock *block, ram_addr_t offset, + int (send_queued_data(CompressParam *))) +{ + int idx, thread_count, pages =3D -1; + bool wait =3D migrate_compress_wait_thread(); + + thread_count =3D migrate_compress_threads(); + qemu_mutex_lock(&comp_done_lock); +retry: + for (idx =3D 0; idx < thread_count; idx++) { + if (comp_param[idx].done) { + CompressParam *param =3D &comp_param[idx]; + qemu_mutex_lock(¶m->mutex); + param->done =3D false; + send_queued_data(param); + compress_reset_result(param); + set_compress_params(param, block, offset); + + qemu_cond_signal(¶m->cond); + qemu_mutex_unlock(¶m->mutex); + pages =3D 1; + break; + } + } + + /* + * wait for the free thread if the user specifies 'compress-wait-threa= d', + * otherwise we will post the page out in the main thread as normal pa= ge. + */ + if (pages < 0 && wait) { + qemu_cond_wait(&comp_done_cond, &comp_done_lock); + goto retry; + } + qemu_mutex_unlock(&comp_done_lock); + + return pages; +} diff --git a/migration/ram-compress.h b/migration/ram-compress.h new file mode 100644 index 0000000000..06570a799c --- /dev/null +++ b/migration/ram-compress.h @@ -0,0 +1,65 @@ +/* + * QEMU System Emulator + * + * Copyright (c) 2003-2008 Fabrice Bellard + * Copyright (c) 2011-2015 Red Hat Inc + * + * Authors: + * Juan Quintela + * + * Permission is hereby granted, free of charge, to any person obtaining a= copy + * of this software and associated documentation files (the "Software"), t= o deal + * in the Software without restriction, including without limitation the r= ights + * to use, copy, modify, merge, publish, distribute, sublicense, and/or se= ll + * copies of the Software, and to permit persons to whom the Software is + * furnished to do so, subject to the following conditions: + * + * The above copyright notice and this permission notice shall be included= in + * all copies or substantial portions of the Software. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS= OR + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL + * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OT= HER + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING= FROM, + * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS = IN + * THE SOFTWARE. + */ + +#ifndef QEMU_MIGRATION_COMPRESS_H +#define QEMU_MIGRATION_COMPRESS_H + +#include "qemu-file.h" + +enum CompressResult { + RES_NONE =3D 0, + RES_ZEROPAGE =3D 1, + RES_COMPRESS =3D 2 +}; +typedef enum CompressResult CompressResult; + +struct CompressParam { + bool done; + bool quit; + bool trigger; + CompressResult result; + QEMUFile *file; + QemuMutex mutex; + QemuCond cond; + RAMBlock *block; + ram_addr_t offset; + + /* internally used fields */ + z_stream stream; + uint8_t *originbuf; +}; +typedef struct CompressParam CompressParam; + +void compress_threads_save_cleanup(void); +int compress_threads_save_setup(void); + +void flush_compressed_data(int (send_queued_data(CompressParam *))); +int compress_page_with_multi_thread(RAMBlock *block, ram_addr_t offset, + int (send_queued_data(CompressParam *))); + +#endif diff --git a/migration/ram.c b/migration/ram.c index 3671912b58..e4e50f5363 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -32,8 +32,8 @@ #include "qemu/bitmap.h" #include "qemu/madvise.h" #include "qemu/main-loop.h" -#include "io/channel-null.h" #include "xbzrle.h" +#include "ram-compress.h" #include "ram.h" #include "migration.h" #include "migration-stats.h" @@ -480,32 +480,6 @@ typedef struct MigrationOps MigrationOps; =20 MigrationOps *migration_ops; =20 -CompressionStats compression_counters; - -enum CompressResult { - RES_NONE =3D 0, - RES_ZEROPAGE =3D 1, - RES_COMPRESS =3D 2 -}; -typedef enum CompressResult CompressResult; - -struct CompressParam { - bool done; - bool quit; - bool trigger; - CompressResult result; - QEMUFile *file; - QemuMutex mutex; - QemuCond cond; - RAMBlock *block; - ram_addr_t offset; - - /* internally used fields */ - z_stream stream; - uint8_t *originbuf; -}; -typedef struct CompressParam CompressParam; - struct DecompressParam { bool done; bool quit; @@ -518,15 +492,6 @@ struct DecompressParam { }; typedef struct DecompressParam DecompressParam; =20 -static CompressParam *comp_param; -static QemuThread *compress_threads; -/* comp_done_cond is used to wake up the migration thread when - * one of the compression threads has finished the compression. - * comp_done_lock is used to co-work with comp_done_cond. - */ -static QemuMutex comp_done_lock; -static QemuCond comp_done_cond; - static QEMUFile *decomp_file; static DecompressParam *decomp_param; static QemuThread *decompress_threads; @@ -535,10 +500,6 @@ static QemuCond decomp_done_cond; =20 static int ram_save_host_page_urgent(PageSearchStatus *pss); =20 -static CompressResult do_compress_ram_page(QEMUFile *f, z_stream *stream, - RAMBlock *block, ram_addr_t off= set, - uint8_t *source_buf); - /* NOTE: page is the PFN not real ram_addr_t. */ static void pss_init(PageSearchStatus *pss, RAMBlock *rb, ram_addr_t page) { @@ -557,123 +518,6 @@ static bool pss_overlap(PageSearchStatus *pss1, PageS= earchStatus *pss2) (pss1->host_page_start =3D=3D pss2->host_page_start); } =20 -static void *do_data_compress(void *opaque) -{ - CompressParam *param =3D opaque; - RAMBlock *block; - ram_addr_t offset; - CompressResult result; - - qemu_mutex_lock(¶m->mutex); - while (!param->quit) { - if (param->trigger) { - block =3D param->block; - offset =3D param->offset; - param->trigger =3D false; - qemu_mutex_unlock(¶m->mutex); - - result =3D do_compress_ram_page(param->file, ¶m->stream, - block, offset, param->originbuf); - - qemu_mutex_lock(&comp_done_lock); - param->done =3D true; - param->result =3D result; - qemu_cond_signal(&comp_done_cond); - qemu_mutex_unlock(&comp_done_lock); - - qemu_mutex_lock(¶m->mutex); - } else { - qemu_cond_wait(¶m->cond, ¶m->mutex); - } - } - qemu_mutex_unlock(¶m->mutex); - - return NULL; -} - -static void compress_threads_save_cleanup(void) -{ - int i, thread_count; - - if (!migrate_compress() || !comp_param) { - return; - } - - thread_count =3D migrate_compress_threads(); - for (i =3D 0; i < thread_count; i++) { - /* - * we use it as a indicator which shows if the thread is - * properly init'd or not - */ - if (!comp_param[i].file) { - break; - } - - qemu_mutex_lock(&comp_param[i].mutex); - comp_param[i].quit =3D true; - qemu_cond_signal(&comp_param[i].cond); - qemu_mutex_unlock(&comp_param[i].mutex); - - qemu_thread_join(compress_threads + i); - qemu_mutex_destroy(&comp_param[i].mutex); - qemu_cond_destroy(&comp_param[i].cond); - deflateEnd(&comp_param[i].stream); - g_free(comp_param[i].originbuf); - qemu_fclose(comp_param[i].file); - comp_param[i].file =3D NULL; - } - qemu_mutex_destroy(&comp_done_lock); - qemu_cond_destroy(&comp_done_cond); - g_free(compress_threads); - g_free(comp_param); - compress_threads =3D NULL; - comp_param =3D NULL; -} - -static int compress_threads_save_setup(void) -{ - int i, thread_count; - - if (!migrate_compress()) { - return 0; - } - thread_count =3D migrate_compress_threads(); - compress_threads =3D g_new0(QemuThread, thread_count); - comp_param =3D g_new0(CompressParam, thread_count); - qemu_cond_init(&comp_done_cond); - qemu_mutex_init(&comp_done_lock); - for (i =3D 0; i < thread_count; i++) { - comp_param[i].originbuf =3D g_try_malloc(TARGET_PAGE_SIZE); - if (!comp_param[i].originbuf) { - goto exit; - } - - if (deflateInit(&comp_param[i].stream, - migrate_compress_level()) !=3D Z_OK) { - g_free(comp_param[i].originbuf); - goto exit; - } - - /* comp_param[i].file is just used as a dummy buffer to save data, - * set its ops to empty. - */ - comp_param[i].file =3D qemu_file_new_output( - QIO_CHANNEL(qio_channel_null_new())); - comp_param[i].done =3D true; - comp_param[i].quit =3D false; - qemu_mutex_init(&comp_param[i].mutex); - qemu_cond_init(&comp_param[i].cond); - qemu_thread_create(compress_threads + i, "compress", - do_data_compress, comp_param + i, - QEMU_THREAD_JOINABLE); - } - return 0; - -exit: - compress_threads_save_cleanup(); - return -1; -} - /** * save_page_header: write page header to wire * @@ -1461,32 +1305,6 @@ static int ram_save_multifd_page(QEMUFile *file, RAM= Block *block, return 1; } =20 -static CompressResult do_compress_ram_page(QEMUFile *f, z_stream *stream, - RAMBlock *block, ram_addr_t off= set, - uint8_t *source_buf) -{ - uint8_t *p =3D block->host + offset; - int ret; - - if (buffer_is_zero(p, TARGET_PAGE_SIZE)) { - return RES_ZEROPAGE; - } - - /* - * copy it to a internal buffer to avoid it being modified by VM - * so that we can catch up the error during compression and - * decompression - */ - memcpy(source_buf, p, TARGET_PAGE_SIZE); - ret =3D qemu_put_compression_data(f, stream, source_buf, TARGET_PAGE_S= IZE); - if (ret < 0) { - qemu_file_set_error(migrate_get_current()->to_dst_file, ret); - error_report("compressed data failed!"); - return RES_NONE; - } - return RES_COMPRESS; -} - static void update_compress_thread_counts(const CompressParam *param, int bytes_xmit) { @@ -1504,13 +1322,6 @@ update_compress_thread_counts(const CompressParam *p= aram, int bytes_xmit) =20 static bool save_page_use_compression(RAMState *rs); =20 -static inline void compress_reset_result(CompressParam *param) -{ - param->result =3D RES_NONE; - param->block =3D NULL; - param->offset =3D 0; -} - static int send_queued_data(CompressParam *param) { PageSearchStatus *pss =3D &ram_state->pss[RAM_CHANNEL_PRECOPY]; @@ -1545,31 +1356,6 @@ static int send_queued_data(CompressParam *param) return len; } =20 -static void flush_compressed_data(int (send_queued_data(CompressParam *))) -{ - int idx, thread_count; - - thread_count =3D migrate_compress_threads(); - - qemu_mutex_lock(&comp_done_lock); - for (idx =3D 0; idx < thread_count; idx++) { - while (!comp_param[idx].done) { - qemu_cond_wait(&comp_done_cond, &comp_done_lock); - } - } - qemu_mutex_unlock(&comp_done_lock); - - for (idx =3D 0; idx < thread_count; idx++) { - qemu_mutex_lock(&comp_param[idx].mutex); - if (!comp_param[idx].quit) { - CompressParam *param =3D &comp_param[idx]; - send_queued_data(param); - compress_reset_result(param); - } - qemu_mutex_unlock(&comp_param[idx].mutex); - } -} - static void ram_flush_compressed_data(RAMState *rs) { if (!save_page_use_compression(rs)) { @@ -1579,52 +1365,6 @@ static void ram_flush_compressed_data(RAMState *rs) flush_compressed_data(send_queued_data); } =20 -static inline void set_compress_params(CompressParam *param, RAMBlock *blo= ck, - ram_addr_t offset) -{ - param->block =3D block; - param->offset =3D offset; - param->trigger =3D true; -} - -static int compress_page_with_multi_thread(RAMBlock *block, ram_addr_t off= set, - int (send_queued_data(CompressParam *))) -{ - int idx, thread_count, pages =3D -1; - bool wait =3D migrate_compress_wait_thread(); - - thread_count =3D migrate_compress_threads(); - qemu_mutex_lock(&comp_done_lock); -retry: - for (idx =3D 0; idx < thread_count; idx++) { - if (comp_param[idx].done) { - CompressParam *param =3D &comp_param[idx]; - qemu_mutex_lock(¶m->mutex); - param->done =3D false; - send_queued_data(param); - compress_reset_result(param); - set_compress_params(param, block, offset); - - qemu_cond_signal(¶m->cond); - qemu_mutex_unlock(¶m->mutex); - pages =3D 1; - break; - } - } - - /* - * wait for the free thread if the user specifies 'compress-wait-threa= d', - * otherwise we will post the page out in the main thread as normal pa= ge. - */ - if (pages < 0 && wait) { - qemu_cond_wait(&comp_done_cond, &comp_done_lock); - goto retry; - } - qemu_mutex_unlock(&comp_done_lock); - - return pages; -} - #define PAGE_ALL_CLEAN 0 #define PAGE_TRY_AGAIN 1 #define PAGE_DIRTY_FOUND 2 --=20 2.40.0 From nobody Fri May 17 02:03:38 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1682709390; cv=none; d=zohomail.com; s=zohoarc; b=idYEukxJHsmrupCz61HjPPIsW15vlJZPeLjcAnKrNLL8CV0+eIKWdHqShVhOEdE6zZOr8Niu66CVDr2kwMquKuM/UfIEeK+1gBfa6NBAc0JZaO4hCPZiDi7ADLyNx1xg6Ca2NhLityEX40ZxENVxHYS2v95FV7bJ9pUPzyfIy7k= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1682709390; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=Wc1FIyuUrIryObKfdag2h/gPhFR3Lnfz5vm7N2t/9ao=; b=aCY427eQasIoEpTJOvIzpb55nXE9AMnnb/xFKxQx6rDvoiwPdRkrEcFq3mQViimf/kWroeqwYpqhYARNNLnYg6u5qnAuNSAganLTvTKyUU7BMlSa5fPAmyr0d2HXGeDN2l331V253hOpZ9K306/KtvQIvzG7bVnfseoN8s/GEVs= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1682709390297752.2560440270801; Fri, 28 Apr 2023 12:16:30 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1psTWN-0004vX-UL; Fri, 28 Apr 2023 15:12:43 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1psTWM-0004vB-C6 for qemu-devel@nongnu.org; Fri, 28 Apr 2023 15:12:42 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1psTWJ-00031q-8f for qemu-devel@nongnu.org; Fri, 28 Apr 2023 15:12:42 -0400 Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-453-Gs0IEWpNOLS72NxShNMbCw-1; Fri, 28 Apr 2023 15:12:37 -0400 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.rdu2.redhat.com [10.11.54.6]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id D7F6E858F09; Fri, 28 Apr 2023 19:12:36 +0000 (UTC) Received: from secure.mitica (unknown [10.39.193.238]) by smtp.corp.redhat.com (Postfix) with ESMTP id 3EA972166B4F; Fri, 28 Apr 2023 19:12:35 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1682709158; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Wc1FIyuUrIryObKfdag2h/gPhFR3Lnfz5vm7N2t/9ao=; b=MZfkWVfjX3VbBH7WkF07uCY0e3dJ2OKvZc1z6U1xx5T+e7tmqLFbnNm9mFCOUtFxCjmhev klH7YCHWvXScI42Z6yOp5ya2q9YEsDMDYqLInC4gfj8llg4dnrpbZ6782YmXlEiMuPQJEy V5mTVf4YAKLHFS1+DHQ5edDFfMLCNYk= X-MC-Unique: Gs0IEWpNOLS72NxShNMbCw-1 From: Juan Quintela To: qemu-devel@nongnu.org Cc: Leonardo Bras , Juan Quintela , Thomas Huth , Laurent Vivier , Paolo Bonzini , Peter Xu , Lukas Straub , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= Subject: [PULL 18/21] ram.c: Move core decompression code into its own file Date: Fri, 28 Apr 2023 21:12:00 +0200 Message-Id: <20230428191203.39520-19-quintela@redhat.com> In-Reply-To: <20230428191203.39520-1-quintela@redhat.com> References: <20230428191203.39520-1-quintela@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.1 on 10.11.54.6 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=quintela@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -22 X-Spam_score: -2.3 X-Spam_bar: -- X-Spam_report: (-2.3 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.171, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1682709392449100001 From: Lukas Straub No functional changes intended. Signed-off-by: Lukas Straub Reviewed-by: Philippe Mathieu-Daud=C3=A9 Reviewed-by: Juan Quintela Signed-off-by: Juan Quintela --- migration/ram-compress.c | 203 ++++++++++++++++++++++++++++++++++++++ migration/ram-compress.h | 5 + migration/ram.c | 204 --------------------------------------- 3 files changed, 208 insertions(+), 204 deletions(-) diff --git a/migration/ram-compress.c b/migration/ram-compress.c index d9bc67d075..c25562f12d 100644 --- a/migration/ram-compress.c +++ b/migration/ram-compress.c @@ -48,6 +48,24 @@ static QemuThread *compress_threads; static QemuMutex comp_done_lock; static QemuCond comp_done_cond; =20 +struct DecompressParam { + bool done; + bool quit; + QemuMutex mutex; + QemuCond cond; + void *des; + uint8_t *compbuf; + int len; + z_stream stream; +}; +typedef struct DecompressParam DecompressParam; + +static QEMUFile *decomp_file; +static DecompressParam *decomp_param; +static QemuThread *decompress_threads; +static QemuMutex decomp_done_lock; +static QemuCond decomp_done_cond; + static CompressResult do_compress_ram_page(QEMUFile *f, z_stream *stream, RAMBlock *block, ram_addr_t off= set, uint8_t *source_buf); @@ -272,3 +290,188 @@ retry: =20 return pages; } + +/* return the size after decompression, or negative value on error */ +static int +qemu_uncompress_data(z_stream *stream, uint8_t *dest, size_t dest_len, + const uint8_t *source, size_t source_len) +{ + int err; + + err =3D inflateReset(stream); + if (err !=3D Z_OK) { + return -1; + } + + stream->avail_in =3D source_len; + stream->next_in =3D (uint8_t *)source; + stream->avail_out =3D dest_len; + stream->next_out =3D dest; + + err =3D inflate(stream, Z_NO_FLUSH); + if (err !=3D Z_STREAM_END) { + return -1; + } + + return stream->total_out; +} + +static void *do_data_decompress(void *opaque) +{ + DecompressParam *param =3D opaque; + unsigned long pagesize; + uint8_t *des; + int len, ret; + + qemu_mutex_lock(¶m->mutex); + while (!param->quit) { + if (param->des) { + des =3D param->des; + len =3D param->len; + param->des =3D 0; + qemu_mutex_unlock(¶m->mutex); + + pagesize =3D TARGET_PAGE_SIZE; + + ret =3D qemu_uncompress_data(¶m->stream, des, pagesize, + param->compbuf, len); + if (ret < 0 && migrate_get_current()->decompress_error_check) { + error_report("decompress data failed"); + qemu_file_set_error(decomp_file, ret); + } + + qemu_mutex_lock(&decomp_done_lock); + param->done =3D true; + qemu_cond_signal(&decomp_done_cond); + qemu_mutex_unlock(&decomp_done_lock); + + qemu_mutex_lock(¶m->mutex); + } else { + qemu_cond_wait(¶m->cond, ¶m->mutex); + } + } + qemu_mutex_unlock(¶m->mutex); + + return NULL; +} + +int wait_for_decompress_done(void) +{ + int idx, thread_count; + + if (!migrate_compress()) { + return 0; + } + + thread_count =3D migrate_decompress_threads(); + qemu_mutex_lock(&decomp_done_lock); + for (idx =3D 0; idx < thread_count; idx++) { + while (!decomp_param[idx].done) { + qemu_cond_wait(&decomp_done_cond, &decomp_done_lock); + } + } + qemu_mutex_unlock(&decomp_done_lock); + return qemu_file_get_error(decomp_file); +} + +void compress_threads_load_cleanup(void) +{ + int i, thread_count; + + if (!migrate_compress()) { + return; + } + thread_count =3D migrate_decompress_threads(); + for (i =3D 0; i < thread_count; i++) { + /* + * we use it as a indicator which shows if the thread is + * properly init'd or not + */ + if (!decomp_param[i].compbuf) { + break; + } + + qemu_mutex_lock(&decomp_param[i].mutex); + decomp_param[i].quit =3D true; + qemu_cond_signal(&decomp_param[i].cond); + qemu_mutex_unlock(&decomp_param[i].mutex); + } + for (i =3D 0; i < thread_count; i++) { + if (!decomp_param[i].compbuf) { + break; + } + + qemu_thread_join(decompress_threads + i); + qemu_mutex_destroy(&decomp_param[i].mutex); + qemu_cond_destroy(&decomp_param[i].cond); + inflateEnd(&decomp_param[i].stream); + g_free(decomp_param[i].compbuf); + decomp_param[i].compbuf =3D NULL; + } + g_free(decompress_threads); + g_free(decomp_param); + decompress_threads =3D NULL; + decomp_param =3D NULL; + decomp_file =3D NULL; +} + +int compress_threads_load_setup(QEMUFile *f) +{ + int i, thread_count; + + if (!migrate_compress()) { + return 0; + } + + thread_count =3D migrate_decompress_threads(); + decompress_threads =3D g_new0(QemuThread, thread_count); + decomp_param =3D g_new0(DecompressParam, thread_count); + qemu_mutex_init(&decomp_done_lock); + qemu_cond_init(&decomp_done_cond); + decomp_file =3D f; + for (i =3D 0; i < thread_count; i++) { + if (inflateInit(&decomp_param[i].stream) !=3D Z_OK) { + goto exit; + } + + decomp_param[i].compbuf =3D g_malloc0(compressBound(TARGET_PAGE_SI= ZE)); + qemu_mutex_init(&decomp_param[i].mutex); + qemu_cond_init(&decomp_param[i].cond); + decomp_param[i].done =3D true; + decomp_param[i].quit =3D false; + qemu_thread_create(decompress_threads + i, "decompress", + do_data_decompress, decomp_param + i, + QEMU_THREAD_JOINABLE); + } + return 0; +exit: + compress_threads_load_cleanup(); + return -1; +} + +void decompress_data_with_multi_threads(QEMUFile *f, void *host, int len) +{ + int idx, thread_count; + + thread_count =3D migrate_decompress_threads(); + QEMU_LOCK_GUARD(&decomp_done_lock); + while (true) { + for (idx =3D 0; idx < thread_count; idx++) { + if (decomp_param[idx].done) { + decomp_param[idx].done =3D false; + qemu_mutex_lock(&decomp_param[idx].mutex); + qemu_get_buffer(f, decomp_param[idx].compbuf, len); + decomp_param[idx].des =3D host; + decomp_param[idx].len =3D len; + qemu_cond_signal(&decomp_param[idx].cond); + qemu_mutex_unlock(&decomp_param[idx].mutex); + break; + } + } + if (idx < thread_count) { + break; + } else { + qemu_cond_wait(&decomp_done_cond, &decomp_done_lock); + } + } +} diff --git a/migration/ram-compress.h b/migration/ram-compress.h index 06570a799c..6f7fe2f472 100644 --- a/migration/ram-compress.h +++ b/migration/ram-compress.h @@ -62,4 +62,9 @@ void flush_compressed_data(int (send_queued_data(Compress= Param *))); int compress_page_with_multi_thread(RAMBlock *block, ram_addr_t offset, int (send_queued_data(CompressParam *))); =20 +int wait_for_decompress_done(void); +void compress_threads_load_cleanup(void); +int compress_threads_load_setup(QEMUFile *f); +void decompress_data_with_multi_threads(QEMUFile *f, void *host, int len); + #endif diff --git a/migration/ram.c b/migration/ram.c index e4e50f5363..250cb01099 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -480,24 +480,6 @@ typedef struct MigrationOps MigrationOps; =20 MigrationOps *migration_ops; =20 -struct DecompressParam { - bool done; - bool quit; - QemuMutex mutex; - QemuCond cond; - void *des; - uint8_t *compbuf; - int len; - z_stream stream; -}; -typedef struct DecompressParam DecompressParam; - -static QEMUFile *decomp_file; -static DecompressParam *decomp_param; -static QemuThread *decompress_threads; -static QemuMutex decomp_done_lock; -static QemuCond decomp_done_cond; - static int ram_save_host_page_urgent(PageSearchStatus *pss); =20 /* NOTE: page is the PFN not real ram_addr_t. */ @@ -3465,192 +3447,6 @@ void ram_handle_compressed(void *host, uint8_t ch, = uint64_t size) } } =20 -/* return the size after decompression, or negative value on error */ -static int -qemu_uncompress_data(z_stream *stream, uint8_t *dest, size_t dest_len, - const uint8_t *source, size_t source_len) -{ - int err; - - err =3D inflateReset(stream); - if (err !=3D Z_OK) { - return -1; - } - - stream->avail_in =3D source_len; - stream->next_in =3D (uint8_t *)source; - stream->avail_out =3D dest_len; - stream->next_out =3D dest; - - err =3D inflate(stream, Z_NO_FLUSH); - if (err !=3D Z_STREAM_END) { - return -1; - } - - return stream->total_out; -} - -static void *do_data_decompress(void *opaque) -{ - DecompressParam *param =3D opaque; - unsigned long pagesize; - uint8_t *des; - int len, ret; - - qemu_mutex_lock(¶m->mutex); - while (!param->quit) { - if (param->des) { - des =3D param->des; - len =3D param->len; - param->des =3D 0; - qemu_mutex_unlock(¶m->mutex); - - pagesize =3D TARGET_PAGE_SIZE; - - ret =3D qemu_uncompress_data(¶m->stream, des, pagesize, - param->compbuf, len); - if (ret < 0 && migrate_get_current()->decompress_error_check) { - error_report("decompress data failed"); - qemu_file_set_error(decomp_file, ret); - } - - qemu_mutex_lock(&decomp_done_lock); - param->done =3D true; - qemu_cond_signal(&decomp_done_cond); - qemu_mutex_unlock(&decomp_done_lock); - - qemu_mutex_lock(¶m->mutex); - } else { - qemu_cond_wait(¶m->cond, ¶m->mutex); - } - } - qemu_mutex_unlock(¶m->mutex); - - return NULL; -} - -static int wait_for_decompress_done(void) -{ - int idx, thread_count; - - if (!migrate_compress()) { - return 0; - } - - thread_count =3D migrate_decompress_threads(); - qemu_mutex_lock(&decomp_done_lock); - for (idx =3D 0; idx < thread_count; idx++) { - while (!decomp_param[idx].done) { - qemu_cond_wait(&decomp_done_cond, &decomp_done_lock); - } - } - qemu_mutex_unlock(&decomp_done_lock); - return qemu_file_get_error(decomp_file); -} - -static void compress_threads_load_cleanup(void) -{ - int i, thread_count; - - if (!migrate_compress()) { - return; - } - thread_count =3D migrate_decompress_threads(); - for (i =3D 0; i < thread_count; i++) { - /* - * we use it as a indicator which shows if the thread is - * properly init'd or not - */ - if (!decomp_param[i].compbuf) { - break; - } - - qemu_mutex_lock(&decomp_param[i].mutex); - decomp_param[i].quit =3D true; - qemu_cond_signal(&decomp_param[i].cond); - qemu_mutex_unlock(&decomp_param[i].mutex); - } - for (i =3D 0; i < thread_count; i++) { - if (!decomp_param[i].compbuf) { - break; - } - - qemu_thread_join(decompress_threads + i); - qemu_mutex_destroy(&decomp_param[i].mutex); - qemu_cond_destroy(&decomp_param[i].cond); - inflateEnd(&decomp_param[i].stream); - g_free(decomp_param[i].compbuf); - decomp_param[i].compbuf =3D NULL; - } - g_free(decompress_threads); - g_free(decomp_param); - decompress_threads =3D NULL; - decomp_param =3D NULL; - decomp_file =3D NULL; -} - -static int compress_threads_load_setup(QEMUFile *f) -{ - int i, thread_count; - - if (!migrate_compress()) { - return 0; - } - - thread_count =3D migrate_decompress_threads(); - decompress_threads =3D g_new0(QemuThread, thread_count); - decomp_param =3D g_new0(DecompressParam, thread_count); - qemu_mutex_init(&decomp_done_lock); - qemu_cond_init(&decomp_done_cond); - decomp_file =3D f; - for (i =3D 0; i < thread_count; i++) { - if (inflateInit(&decomp_param[i].stream) !=3D Z_OK) { - goto exit; - } - - decomp_param[i].compbuf =3D g_malloc0(compressBound(TARGET_PAGE_SI= ZE)); - qemu_mutex_init(&decomp_param[i].mutex); - qemu_cond_init(&decomp_param[i].cond); - decomp_param[i].done =3D true; - decomp_param[i].quit =3D false; - qemu_thread_create(decompress_threads + i, "decompress", - do_data_decompress, decomp_param + i, - QEMU_THREAD_JOINABLE); - } - return 0; -exit: - compress_threads_load_cleanup(); - return -1; -} - -static void decompress_data_with_multi_threads(QEMUFile *f, - void *host, int len) -{ - int idx, thread_count; - - thread_count =3D migrate_decompress_threads(); - QEMU_LOCK_GUARD(&decomp_done_lock); - while (true) { - for (idx =3D 0; idx < thread_count; idx++) { - if (decomp_param[idx].done) { - decomp_param[idx].done =3D false; - qemu_mutex_lock(&decomp_param[idx].mutex); - qemu_get_buffer(f, decomp_param[idx].compbuf, len); - decomp_param[idx].des =3D host; - decomp_param[idx].len =3D len; - qemu_cond_signal(&decomp_param[idx].cond); - qemu_mutex_unlock(&decomp_param[idx].mutex); - break; - } - } - if (idx < thread_count) { - break; - } else { - qemu_cond_wait(&decomp_done_cond, &decomp_done_lock); - } - } -} - static void colo_init_ram_state(void) { ram_state_init(&ram_state); --=20 2.40.0 From nobody Fri May 17 02:03:38 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1682709370; cv=none; d=zohomail.com; s=zohoarc; b=PAswkkW3rG7GV6HcNFMYDTyVy++kYxY4u79F6/gl4kbboUuyRuREpKZpQdxYTIxbqObfSc6Sso4dqkkqk8643xkax28nq9ROCUbvZa9FJ36dv4ijtFqKtm20qRwXhaLtFfuX4yu4fGgzPKOeTAx1ZMH3/ySbSBtOMvaXQHMFtlU= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1682709370; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=BZNrAx/yY1mGvZnXcB5fz0XHo3uEWqvLNFcOkMtbtbg=; b=LMiJGpI8kE1rV5Ca+Cs86fvZdm+yXwwppedEinPzJSFKB6LkYxh+JgqnIfxQEwfSvtfR5d/+QiRwDV494sT9L3xq7pdCm/RFeHeD3XhxisPNgcXsRIvR5bNb4rwZRTGCnnLN+Ce8NHqRmZg+QIldQ80kCPQ2qJBcjUQhFQzxmEs= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1682709370375796.8501599065332; Fri, 28 Apr 2023 12:16:10 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1psTWQ-0004wI-Hx; Fri, 28 Apr 2023 15:12:46 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1psTWP-0004w4-Cf for qemu-devel@nongnu.org; Fri, 28 Apr 2023 15:12:45 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1psTWN-00032B-PT for qemu-devel@nongnu.org; Fri, 28 Apr 2023 15:12:45 -0400 Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-13-TE7FmrVZPRmHRkV3Vh9jYQ-1; Fri, 28 Apr 2023 15:12:38 -0400 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.rdu2.redhat.com [10.11.54.6]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 8E4098314E9; Fri, 28 Apr 2023 19:12:38 +0000 (UTC) Received: from secure.mitica (unknown [10.39.193.238]) by smtp.corp.redhat.com (Postfix) with ESMTP id 3CCDB2166B4F; Fri, 28 Apr 2023 19:12:37 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1682709162; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=BZNrAx/yY1mGvZnXcB5fz0XHo3uEWqvLNFcOkMtbtbg=; b=Mxn5akFM5SsvRDi6JQ/I6/Id0ACEu0P7oWdMeBaLykHT/X3qLHkt9D4fAAb2lJ0bI2T17s ErOykNbVST9IOykdR2NP57GcPy6CvY9Yn41OTn3BlsmQoeEqZyNpF36QqgYDUKkmyqnXw2 wxFLxMUwcU/6aMMuHvsAZN6X7/sfOhQ= X-MC-Unique: TE7FmrVZPRmHRkV3Vh9jYQ-1 From: Juan Quintela To: qemu-devel@nongnu.org Cc: Leonardo Bras , Juan Quintela , Thomas Huth , Laurent Vivier , Paolo Bonzini , Peter Xu , Lukas Straub Subject: [PULL 19/21] ram compress: Assert that the file buffer matches the result Date: Fri, 28 Apr 2023 21:12:01 +0200 Message-Id: <20230428191203.39520-20-quintela@redhat.com> In-Reply-To: <20230428191203.39520-1-quintela@redhat.com> References: <20230428191203.39520-1-quintela@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.1 on 10.11.54.6 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=quintela@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -22 X-Spam_score: -2.3 X-Spam_bar: -- X-Spam_report: (-2.3 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.171, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1682709371734100001 Content-Type: text/plain; charset="utf-8" From: Lukas Straub Before this series, "nothing to send" was handled by the file buffer being empty. Now it is tracked via param->result. Assert that the file buffer state matches the result. Signed-off-by: Lukas Straub Reviewed-by: Juan Quintela Signed-off-by: Juan Quintela --- migration/qemu-file.c | 11 +++++++++++ migration/qemu-file.h | 1 + migration/ram-compress.c | 5 +++++ migration/ram.c | 2 ++ 4 files changed, 19 insertions(+) diff --git a/migration/qemu-file.c b/migration/qemu-file.c index ee04240a21..9c1ef8e3f9 100644 --- a/migration/qemu-file.c +++ b/migration/qemu-file.c @@ -887,6 +887,17 @@ int qemu_put_qemu_file(QEMUFile *f_des, QEMUFile *f_sr= c) return len; } =20 +/* + * Check if the writable buffer is empty + */ + +bool qemu_file_buffer_empty(QEMUFile *file) +{ + assert(qemu_file_is_writable(file)); + + return !file->iovcnt; +} + /* * Get a string whose length is determined by a single preceding byte * A preallocated 256 byte buffer must be passed in. diff --git a/migration/qemu-file.h b/migration/qemu-file.h index d16cd50448..c2d6086a6e 100644 --- a/migration/qemu-file.h +++ b/migration/qemu-file.h @@ -113,6 +113,7 @@ size_t coroutine_mixed_fn qemu_get_buffer_in_place(QEMU= File *f, uint8_t **buf, s ssize_t qemu_put_compression_data(QEMUFile *f, z_stream *stream, const uint8_t *p, size_t size); int qemu_put_qemu_file(QEMUFile *f_des, QEMUFile *f_src); +bool qemu_file_buffer_empty(QEMUFile *file); =20 /* * Note that you can only peek continuous bytes from where the current poi= nter diff --git a/migration/ram-compress.c b/migration/ram-compress.c index c25562f12d..3d2a4a6329 100644 --- a/migration/ram-compress.c +++ b/migration/ram-compress.c @@ -194,6 +194,8 @@ static CompressResult do_compress_ram_page(QEMUFile *f,= z_stream *stream, uint8_t *p =3D block->host + offset; int ret; =20 + assert(qemu_file_buffer_empty(f)); + if (buffer_is_zero(p, TARGET_PAGE_SIZE)) { return RES_ZEROPAGE; } @@ -208,6 +210,7 @@ static CompressResult do_compress_ram_page(QEMUFile *f,= z_stream *stream, if (ret < 0) { qemu_file_set_error(migrate_get_current()->to_dst_file, ret); error_report("compressed data failed!"); + qemu_fflush(f); return RES_NONE; } return RES_COMPRESS; @@ -239,6 +242,7 @@ void flush_compressed_data(int (send_queued_data(Compre= ssParam *))) if (!comp_param[idx].quit) { CompressParam *param =3D &comp_param[idx]; send_queued_data(param); + assert(qemu_file_buffer_empty(param->file)); compress_reset_result(param); } qemu_mutex_unlock(&comp_param[idx].mutex); @@ -268,6 +272,7 @@ retry: qemu_mutex_lock(¶m->mutex); param->done =3D false; send_queued_data(param); + assert(qemu_file_buffer_empty(param->file)); compress_reset_result(param); set_compress_params(param, block, offset); =20 diff --git a/migration/ram.c b/migration/ram.c index 250cb01099..a5bd943cc6 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -1321,11 +1321,13 @@ static int send_queued_data(CompressParam *param) assert(block =3D=3D pss->last_sent_block); =20 if (param->result =3D=3D RES_ZEROPAGE) { + assert(qemu_file_buffer_empty(param->file)); len +=3D save_page_header(pss, file, block, offset | RAM_SAVE_FLAG= _ZERO); qemu_put_byte(file, 0); len +=3D 1; ram_release_page(block->idstr, offset); } else if (param->result =3D=3D RES_COMPRESS) { + assert(!qemu_file_buffer_empty(param->file)); len +=3D save_page_header(pss, file, block, offset | RAM_SAVE_FLAG_COMPRESS_PAGE); len +=3D qemu_put_qemu_file(file, param->file); --=20 2.40.0 From nobody Fri May 17 02:03:38 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1682709490; cv=none; d=zohomail.com; s=zohoarc; b=WCau5ejc5VKx9wM64JmntAGI6fUfa/nUXe07APMDzH1dZt5Lk6zMlqzZ1EWZDZ76aejaPsVUU5K+8IUR9PgIvyAyqqKhkqbahFqUtWr3ForNsXDXqIXUIHK7quuda/RL5osKCJl6tc1As2N3ROzgOnQfUU36AkG7xkJ7iadC2Cs= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1682709490; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=cqRiOcVsnq1dkPr7YEiysrQqEMeOB41ddSA57KdHrJY=; b=DYyb1fJY9u1S8P34fOpnsNslIkcudRhZrWa3pUif+a2QHmL+yGzuP8WllAPTWLU9ba3S9KKu/Zy2jHK+m5Op6ghtabNwX79qNhmvUHt6XDih9KyhwEwGQqAttHOCqdReiHkr5tpiv2nRDshdNI+2Kz0BVOgg8GLKsrUClLwurew= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1682709490083792.7143909137219; Fri, 28 Apr 2023 12:18:10 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1psTWb-000571-JR; Fri, 28 Apr 2023 15:12:57 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1psTWT-0004yB-8h for qemu-devel@nongnu.org; Fri, 28 Apr 2023 15:12:49 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1psTWO-00032H-H8 for qemu-devel@nongnu.org; Fri, 28 Apr 2023 15:12:48 -0400 Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-467-z9T1PqOENKWqlOqtgY3HdQ-1; Fri, 28 Apr 2023 15:12:40 -0400 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.rdu2.redhat.com [10.11.54.6]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 529EB1C06913; Fri, 28 Apr 2023 19:12:40 +0000 (UTC) Received: from secure.mitica (unknown [10.39.193.238]) by smtp.corp.redhat.com (Postfix) with ESMTP id D33182166B4F; Fri, 28 Apr 2023 19:12:38 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1682709164; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=cqRiOcVsnq1dkPr7YEiysrQqEMeOB41ddSA57KdHrJY=; b=AxHLs/YzfjgaAgaVO50GvhtQRf3LrAzZyHInWcsszq0a2t8eWHbCoGtxPus3OKLFVh8NPy oo2VTFGvYP5XRIgywMnsyuCtZXc67WFE7fu+saj7aLN8O9OzOP5CpO64XZKDXb74arphQ4 +1vREh6KabeIR3iFdY4SIpaOBmslBaQ= X-MC-Unique: z9T1PqOENKWqlOqtgY3HdQ-1 From: Juan Quintela To: qemu-devel@nongnu.org Cc: Leonardo Bras , Juan Quintela , Thomas Huth , Laurent Vivier , Paolo Bonzini , Peter Xu , Lukas Straub , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= Subject: [PULL 20/21] ram-compress.c: Make target independent Date: Fri, 28 Apr 2023 21:12:02 +0200 Message-Id: <20230428191203.39520-21-quintela@redhat.com> In-Reply-To: <20230428191203.39520-1-quintela@redhat.com> References: <20230428191203.39520-1-quintela@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.1 on 10.11.54.6 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=quintela@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -22 X-Spam_score: -2.3 X-Spam_bar: -- X-Spam_report: (-2.3 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.171, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1682709491974100003 From: Lukas Straub Make ram-compress.c target independent. Signed-off-by: Lukas Straub Reviewed-by: Philippe Mathieu-Daud=C3=A9 Reviewed-by: Juan Quintela Signed-off-by: Juan Quintela --- migration/meson.build | 3 ++- migration/ram-compress.c | 17 ++++++++++------- 2 files changed, 12 insertions(+), 8 deletions(-) diff --git a/migration/meson.build b/migration/meson.build index 2090af8e85..75de868bb7 100644 --- a/migration/meson.build +++ b/migration/meson.build @@ -23,6 +23,8 @@ softmmu_ss.add(files( 'migration.c', 'multifd.c', 'multifd-zlib.c', + 'multifd-zlib.c', + 'ram-compress.c', 'options.c', 'postcopy-ram.c', 'savevm.c', @@ -40,5 +42,4 @@ softmmu_ss.add(when: zstd, if_true: files('multifd-zstd.c= ')) specific_ss.add(when: 'CONFIG_SOFTMMU', if_true: files('dirtyrate.c', 'ram.c', - 'ram-compress.c', 'target.c')) diff --git a/migration/ram-compress.c b/migration/ram-compress.c index 3d2a4a6329..06254d8c69 100644 --- a/migration/ram-compress.c +++ b/migration/ram-compress.c @@ -35,7 +35,8 @@ #include "migration.h" #include "options.h" #include "io/channel-null.h" -#include "exec/ram_addr.h" +#include "exec/target_page.h" +#include "exec/ramblock.h" =20 CompressionStats compression_counters; =20 @@ -156,7 +157,7 @@ int compress_threads_save_setup(void) qemu_cond_init(&comp_done_cond); qemu_mutex_init(&comp_done_lock); for (i =3D 0; i < thread_count; i++) { - comp_param[i].originbuf =3D g_try_malloc(TARGET_PAGE_SIZE); + comp_param[i].originbuf =3D g_try_malloc(qemu_target_page_size()); if (!comp_param[i].originbuf) { goto exit; } @@ -192,11 +193,12 @@ static CompressResult do_compress_ram_page(QEMUFile *= f, z_stream *stream, uint8_t *source_buf) { uint8_t *p =3D block->host + offset; + size_t page_size =3D qemu_target_page_size(); int ret; =20 assert(qemu_file_buffer_empty(f)); =20 - if (buffer_is_zero(p, TARGET_PAGE_SIZE)) { + if (buffer_is_zero(p, page_size)) { return RES_ZEROPAGE; } =20 @@ -205,8 +207,8 @@ static CompressResult do_compress_ram_page(QEMUFile *f,= z_stream *stream, * so that we can catch up the error during compression and * decompression */ - memcpy(source_buf, p, TARGET_PAGE_SIZE); - ret =3D qemu_put_compression_data(f, stream, source_buf, TARGET_PAGE_S= IZE); + memcpy(source_buf, p, page_size); + ret =3D qemu_put_compression_data(f, stream, source_buf, page_size); if (ret < 0) { qemu_file_set_error(migrate_get_current()->to_dst_file, ret); error_report("compressed data failed!"); @@ -336,7 +338,7 @@ static void *do_data_decompress(void *opaque) param->des =3D 0; qemu_mutex_unlock(¶m->mutex); =20 - pagesize =3D TARGET_PAGE_SIZE; + pagesize =3D qemu_target_page_size(); =20 ret =3D qemu_uncompress_data(¶m->stream, des, pagesize, param->compbuf, len); @@ -439,7 +441,8 @@ int compress_threads_load_setup(QEMUFile *f) goto exit; } =20 - decomp_param[i].compbuf =3D g_malloc0(compressBound(TARGET_PAGE_SI= ZE)); + size_t compbuf_size =3D compressBound(qemu_target_page_size()); + decomp_param[i].compbuf =3D g_malloc0(compbuf_size); qemu_mutex_init(&decomp_param[i].mutex); qemu_cond_init(&decomp_param[i].cond); decomp_param[i].done =3D true; --=20 2.40.0 From nobody Fri May 17 02:03:38 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1682709342; cv=none; d=zohomail.com; s=zohoarc; b=MDpCQH40MJYvrXICbfMrzgw1yJR/7jG6+xk+E93k9xD9tO14RhS2JLoElyg+MczLIkd+ZOoUbY1RPX+uN+TRNiJgp87N0kIoMeYcxHL+2DiWJP7EumT/Cns5B3jO1MvvjwlYgW+YG4HQQiLHy3WXK4AXAvdYsc7GgN1XhakZzrA= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1682709342; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=ODr1+q+whZ7g0He07Wyb5qTASgn+dGPdmIiuYcQElcM=; b=QQtL2/Fkpg8ddPCfl54+uDh/fF1d3fHb7C5WolnYJ77ycTOO1YHlnyFyKl0wxGBqkXceCdixymiKQclb//cuh/gA5BFqO2Y6VhRBM8G3MyimWSqXfoltAdGAjnBpuk+sBwyy0SwGNfKPgLMRC6c/WcjLVVlkKV0dIlWAjOb3T54= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1682709342844800.408152434968; Fri, 28 Apr 2023 12:15:42 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1psTWU-0004yE-15; Fri, 28 Apr 2023 15:12:50 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1psTWR-0004xM-Lv for qemu-devel@nongnu.org; Fri, 28 Apr 2023 15:12:47 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1psTWQ-00032d-4y for qemu-devel@nongnu.org; Fri, 28 Apr 2023 15:12:47 -0400 Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-184-n7pJJhklM0Wda0AmD0PrVw-1; Fri, 28 Apr 2023 15:12:42 -0400 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.rdu2.redhat.com [10.11.54.6]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id F23EC1C06911; Fri, 28 Apr 2023 19:12:41 +0000 (UTC) Received: from secure.mitica (unknown [10.39.193.238]) by smtp.corp.redhat.com (Postfix) with ESMTP id 96A5D2166BA0; Fri, 28 Apr 2023 19:12:40 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1682709165; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=ODr1+q+whZ7g0He07Wyb5qTASgn+dGPdmIiuYcQElcM=; b=Raz8w0f6cqYK5rDyZ66AqtxoyKCCSES7wJAwexTh9SMc6p0TRX/2BLjCDh2t+ZEnzxumm5 evm2CMsVZh0zVUJO2hJtm0aQ9zNtLDkhggLjOngMX0AUeL3vf/0Kvtcl8CUXFFr1SkTMxD CFO5OuP0wUvT1/AXPlo/5UPoP08ouCA= X-MC-Unique: n7pJJhklM0Wda0AmD0PrVw-1 From: Juan Quintela To: qemu-devel@nongnu.org Cc: Leonardo Bras , Juan Quintela , Thomas Huth , Laurent Vivier , Paolo Bonzini , Peter Xu , Lukas Straub Subject: [PULL 21/21] migration: Initialize and cleanup decompression in migration.c Date: Fri, 28 Apr 2023 21:12:03 +0200 Message-Id: <20230428191203.39520-22-quintela@redhat.com> In-Reply-To: <20230428191203.39520-1-quintela@redhat.com> References: <20230428191203.39520-1-quintela@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.1 on 10.11.54.6 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=quintela@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -22 X-Spam_score: -2.3 X-Spam_bar: -- X-Spam_report: (-2.3 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.171, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1682709343261100001 Content-Type: text/plain; charset="utf-8" From: Lukas Straub This fixes compress with colo. Signed-off-by: Lukas Straub Reviewed-by: Juan Quintela Signed-off-by: Juan Quintela --- migration/migration.c | 9 +++++++++ migration/ram.c | 5 ----- 2 files changed, 9 insertions(+), 5 deletions(-) diff --git a/migration/migration.c b/migration/migration.c index feb5ab7493..11f5a2dd6f 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -26,6 +26,7 @@ #include "sysemu/cpu-throttle.h" #include "rdma.h" #include "ram.h" +#include "ram-compress.h" #include "migration/global_state.h" #include "migration/misc.h" #include "migration.h" @@ -228,6 +229,7 @@ void migration_incoming_state_destroy(void) struct MigrationIncomingState *mis =3D migration_incoming_get_current(= ); =20 multifd_load_cleanup(); + compress_threads_load_cleanup(); =20 if (mis->to_src_file) { /* Tell source that we are done */ @@ -500,6 +502,12 @@ process_incoming_migration_co(void *opaque) Error *local_err =3D NULL; =20 assert(mis->from_src_file); + + if (compress_threads_load_setup(mis->from_src_file)) { + error_report("Failed to setup decompress threads"); + goto fail; + } + mis->migration_incoming_co =3D qemu_coroutine_self(); mis->largest_page_size =3D qemu_ram_pagesize_largest(); postcopy_state_set(POSTCOPY_INCOMING_NONE); @@ -565,6 +573,7 @@ fail: qemu_fclose(mis->from_src_file); =20 multifd_load_cleanup(); + compress_threads_load_cleanup(); =20 exit(EXIT_FAILURE); } diff --git a/migration/ram.c b/migration/ram.c index a5bd943cc6..1f664fa071 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -3558,10 +3558,6 @@ void colo_release_ram_cache(void) */ static int ram_load_setup(QEMUFile *f, void *opaque) { - if (compress_threads_load_setup(f)) { - return -1; - } - xbzrle_load_setup(); ramblock_recv_map_init(); =20 @@ -3577,7 +3573,6 @@ static int ram_load_cleanup(void *opaque) } =20 xbzrle_load_cleanup(); - compress_threads_load_cleanup(); =20 RAMBLOCK_FOREACH_NOT_IGNORED(rb) { g_free(rb->receivedmap); --=20 2.40.0