From nobody Mon May 20 23:52:52 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1634635890938522.6851677931088; Tue, 19 Oct 2021 02:31:30 -0700 (PDT) Received: from localhost ([::1]:39732 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mclSz-00069A-RI for importer@patchew.org; Tue, 19 Oct 2021 05:31:29 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:54704) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mclQr-0003dI-4G for qemu-devel@nongnu.org; Tue, 19 Oct 2021 05:29:17 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:26519) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mclQp-0005c9-Ip for qemu-devel@nongnu.org; Tue, 19 Oct 2021 05:29:16 -0400 Received: from mail-wm1-f70.google.com (mail-wm1-f70.google.com [209.85.128.70]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-270-8li7iqUlNIOO3G5OBZXjvg-1; Tue, 19 Oct 2021 05:29:13 -0400 Received: by mail-wm1-f70.google.com with SMTP id y142-20020a1c7d94000000b00322f2e380f2so542574wmc.6 for ; Tue, 19 Oct 2021 02:29:13 -0700 (PDT) Received: from localhost (static-233-86-86-188.ipcom.comunitel.net. [188.86.86.233]) by smtp.gmail.com with ESMTPSA id k27sm1717984wms.21.2021.10.19.02.29.10 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 19 Oct 2021 02:29:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1634635754; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=XOX7nqARL3VBZUhOlMwCyMghOR6R49bsh/b43icc/og=; b=c5ukPO0a6rUZqlPnGmU6R77xZYnhK7/WCzSl2HDaVhD2Jy5vRFn32pqyfjsUP7BkLyd9+O LGCVtQZgtafxmWHKFx7kkpjU/WHjoZxfKWv6i6jeqdIMUIvj0zbhad3ekEzIGh7qqFp7RM DMi8h+YBwKeYcP6bWbyBBbtcL51aqrA= X-MC-Unique: 8li7iqUlNIOO3G5OBZXjvg-1 X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=XOX7nqARL3VBZUhOlMwCyMghOR6R49bsh/b43icc/og=; b=Qd6dzp9F5VwaTpo16fr/f27FyVnyiPTohPiUXJyfphnetOMyJSdWmfrkwTYtlQDG8O MkXNwuLMUQZpsSTK/+K06/Vej99rppxxda14uXDMoH5SpezLjW4my00At8Cl1ZrompZw yZaAOhW3ydwuOyMexq5Fo5uga8n/GBlLPbEj/mHtoUgkavXSJJwrtzfLKIN7B9pQr1Dx Im6eh82OmThur1UmbEic9s+P+tkBECnQTz5G5Htux5dfHt891pzlShuImAr/w/8wfrWk oX1K5cqMT0+qcU6gvFs9eNP6hp2iMRbt2vogFHR58BUDjsri3ftwGxCe+9Job3zY2yJ5 /tmg== X-Gm-Message-State: AOAM531thkWLA+muOkJicaI9zj+IckQSqIJJluPeG8TXkDv8PEDlp8OS /w5HmQisxJ+mbWHTuLZFw3BKwJAhEo7IxtrTpnLbhLjdBDRIx5s4I1qd+Ai+dYfOFeod3Qg8cJW zwCNJgU4QbgjaEpUoYILZwbTwPTx66uyRT8+12EBhf1egfXzXIgqF5w1IjZUHUuqqEcg= X-Received: by 2002:a5d:58c1:: with SMTP id o1mr25586285wrf.181.1634635751910; Tue, 19 Oct 2021 02:29:11 -0700 (PDT) X-Google-Smtp-Source: ABdhPJwcgWt6Wy2h78Dt3EjrNXMy3t9CdHC8VdiKtKJKnYlbx1NwV8JODdn0l/WUlpIGetzgSkDHig== X-Received: by 2002:a5d:58c1:: with SMTP id o1mr25586249wrf.181.1634635751560; Tue, 19 Oct 2021 02:29:11 -0700 (PDT) From: Juan Quintela To: qemu-devel@nongnu.org Subject: [PULL 1/7] multifd: Implement yank for multifd send side Date: Tue, 19 Oct 2021 11:29:01 +0200 Message-Id: <20211019092907.5255-2-quintela@redhat.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20211019092907.5255-1-quintela@redhat.com> References: <20211019092907.5255-1-quintela@redhat.com> MIME-Version: 1.0 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=quintela@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=quintela@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -28 X-Spam_score: -2.9 X-Spam_bar: -- X-Spam_report: (-2.9 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.049, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Lukas Straub , "Dr. David Alan Gilbert" , Leonardo Bras , Juan Quintela Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZM-MESSAGEID: 1634635892893100001 Content-Type: text/plain; charset="utf-8" From: Lukas Straub To: qemu-devel Cc: "Dr. David Alan Gilbert" , Juan Quintela , Peter Xu , Leonardo Bras Soares Passos Date: Wed, 1 Sep 2021 17:58:57 +0200 (1 week, 15 hours, 17 minutes ago) [[PGP Signed Part:No public key for 35AB0B289C5DB258 created at 2021-09-01T= 17:58:57+0200 using RSA]] When introducing yank functionality in the migration code I forgot to cover the multifd send side. Signed-off-by: Lukas Straub Tested-by: Leonardo Bras Reviewed-by: Leonardo Bras Reviewed-by: Juan Quintela Signed-off-by: Juan Quintela --- migration/multifd.h | 2 ++ migration/multifd.c | 6 +++++- 2 files changed, 7 insertions(+), 1 deletion(-) diff --git a/migration/multifd.h b/migration/multifd.h index 8d6751f5ed..16c4d112d1 100644 --- a/migration/multifd.h +++ b/migration/multifd.h @@ -85,6 +85,8 @@ typedef struct { bool running; /* should this thread finish */ bool quit; + /* is the yank function registered */ + bool registered_yank; /* thread has work to do */ int pending_job; /* array of pages to sent */ diff --git a/migration/multifd.c b/migration/multifd.c index 377da78f5b..5a4f158f3c 100644 --- a/migration/multifd.c +++ b/migration/multifd.c @@ -546,6 +546,9 @@ void multifd_save_cleanup(void) MultiFDSendParams *p =3D &multifd_send_state->params[i]; Error *local_err =3D NULL; =20 + if (p->registered_yank) { + migration_ioc_unregister_yank(p->c); + } socket_send_channel_destroy(p->c); p->c =3D NULL; qemu_mutex_destroy(&p->mutex); @@ -813,7 +816,8 @@ static bool multifd_channel_connect(MultiFDSendParams *= p, return false; } } else { - /* update for tls qio channel */ + migration_ioc_register_yank(ioc); + p->registered_yank =3D true; p->c =3D ioc; qemu_thread_create(&p->thread, p->name, multifd_send_thread, p, QEMU_THREAD_JOINABLE); --=20 2.31.1 From nobody Mon May 20 23:52:52 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1634636071125114.41645432575001; Tue, 19 Oct 2021 02:34:31 -0700 (PDT) Received: from localhost ([::1]:47852 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mclVu-0003Lg-3N for importer@patchew.org; Tue, 19 Oct 2021 05:34:30 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:54734) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mclQt-0003fx-La for qemu-devel@nongnu.org; Tue, 19 Oct 2021 05:29:19 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:33590) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mclQr-0005cq-2W for qemu-devel@nongnu.org; Tue, 19 Oct 2021 05:29:19 -0400 Received: from mail-wm1-f69.google.com (mail-wm1-f69.google.com [209.85.128.69]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-435-oMp63k84Mf27oS4WNamWOw-1; Tue, 19 Oct 2021 05:29:14 -0400 Received: by mail-wm1-f69.google.com with SMTP id n9-20020a1c7209000000b0030da7d466b8so912376wmc.5 for ; Tue, 19 Oct 2021 02:29:14 -0700 (PDT) Received: from localhost (static-233-86-86-188.ipcom.comunitel.net. [188.86.86.233]) by smtp.gmail.com with ESMTPSA id s186sm2145799wme.14.2021.10.19.02.29.12 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 19 Oct 2021 02:29:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1634635755; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=loashz0+j0OcIJRCFBlcKJzh8A8l4zO8lqCJ5F9pqk4=; b=gYYQQgyQIoYht2j9YBnKBshzAE6V5AmA/4D8ylO/dyKwuCpAKfQIpQwUAuxzU6Eail6BD0 AYLk/BoQm/WqC1P/FeuD9FgZDHVWOXpGbColIyoHvvSYIXA9Bj3qJI9uKUITyKL3MHTCFp 6NOW7ciJVUpYTgvHl1tNZisuhqp7bq8= X-MC-Unique: oMp63k84Mf27oS4WNamWOw-1 X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=loashz0+j0OcIJRCFBlcKJzh8A8l4zO8lqCJ5F9pqk4=; b=vc8nloWYN+DpRDMog+DqB+GqP6YfYoIDHAVk4MQDLl7/a5HbyjAmW/1/8DaFvNpigW JAw9sN9pr3aQhSQ3zgLXhRMvH2Ils+mtIBLiHLHOpf09KQt8ao/57j8ht19bSx7tBCRF 62IMC1tXt/DSxlZuSFjnfyl5cqB1gkAufaT7c90btLfoXDvD8VYUhOgwpODtYvArI3i4 7N6uUCwDHY0wh8AuzEOnjw4DWl67AHIFuTenF9IXIMiEB9OP2w6dn6zk+F/87klcQwqv E691yClUM3BuFtwt3RigrIP1VP0NDkie7YG2UAVN3wrcSLUWiYoQZvUDKamd/9j+SsAE 2jOg== X-Gm-Message-State: AOAM532XWDKScz6FNzTl2AfuMSpiU8FZkcx4//KuLVkVKT281uoHAJYe Yx2cBSDn0pn5oCO4zyaiv2uE4fO/s21sx8iW3ACXLhV76/wQpfk44WubHebq8lzDMKljQ8qzRIA ML7Wx+iLbLPopdbTw48UF7EvASaCKg5vvhxrbI+/yZqNFUbLRPPfh5y/vu2FZBFug8d8= X-Received: by 2002:adf:d1c9:: with SMTP id b9mr43372917wrd.426.1634635752864; Tue, 19 Oct 2021 02:29:12 -0700 (PDT) X-Google-Smtp-Source: ABdhPJzaIQbT8pVgzX7S3gPoba7EXVmfLRi/jYrziEaoQg1DUbL6+eWZxFMnXXzX3zrb7i3NkBGJ4Q== X-Received: by 2002:adf:d1c9:: with SMTP id b9mr43372895wrd.426.1634635752690; Tue, 19 Oct 2021 02:29:12 -0700 (PDT) From: Juan Quintela To: qemu-devel@nongnu.org Subject: [PULL 2/7] multifd: Unconditionally unregister yank function Date: Tue, 19 Oct 2021 11:29:02 +0200 Message-Id: <20211019092907.5255-3-quintela@redhat.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20211019092907.5255-1-quintela@redhat.com> References: <20211019092907.5255-1-quintela@redhat.com> MIME-Version: 1.0 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=quintela@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=216.205.24.124; envelope-from=quintela@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -28 X-Spam_score: -2.9 X-Spam_bar: -- X-Spam_report: (-2.9 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.049, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Lukas Straub , "Dr. David Alan Gilbert" , Juan Quintela Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZM-MESSAGEID: 1634636072377100001 Content-Type: text/plain; charset="utf-8" From: Lukas Straub To: qemu-devel Cc: "Dr. David Alan Gilbert" , Juan Quintela , Peter Xu , Leonardo Bras Soares Passos Date: Wed, 4 Aug 2021 21:26:32 +0200 (5 weeks, 11 hours, 52 minutes ago) [[PGP Signed Part:No public key for 35AB0B289C5DB258 created at 2021-08-04T= 21:26:32+0200 using RSA]] Unconditionally unregister yank function in multifd_load_cleanup(). If it is not unregistered here, it will leak and cause a crash in yank_unregister_instance(). Now if the ioc is still in use afterwards, it will only lead to qemu not being able to recover from a hang related to that ioc. After checking the code, i am pretty sure that ref is always 1 when arriving here. So all this currently does is remove the unneeded check. Signed-off-by: Lukas Straub Reviewed-by: Juan Quintela Signed-off-by: Juan Quintela --- migration/multifd.c | 5 +---- 1 file changed, 1 insertion(+), 4 deletions(-) diff --git a/migration/multifd.c b/migration/multifd.c index 5a4f158f3c..efd424bc97 100644 --- a/migration/multifd.c +++ b/migration/multifd.c @@ -991,10 +991,7 @@ int multifd_load_cleanup(Error **errp) for (i =3D 0; i < migrate_multifd_channels(); i++) { MultiFDRecvParams *p =3D &multifd_recv_state->params[i]; =20 - if (OBJECT(p->c)->ref =3D=3D 1) { - migration_ioc_unregister_yank(p->c); - } - + migration_ioc_unregister_yank(p->c); object_unref(OBJECT(p->c)); p->c =3D NULL; qemu_mutex_destroy(&p->mutex); --=20 2.31.1 From nobody Mon May 20 23:52:52 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1634636264056331.8511479608537; Tue, 19 Oct 2021 02:37:44 -0700 (PDT) Received: from localhost ([::1]:55086 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mclZ1-0008MA-0W for importer@patchew.org; Tue, 19 Oct 2021 05:37:43 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:54850) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mclR7-0004PO-Ko for qemu-devel@nongnu.org; Tue, 19 Oct 2021 05:29:34 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:55990) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mclQr-0005dZ-9d for qemu-devel@nongnu.org; Tue, 19 Oct 2021 05:29:33 -0400 Received: from mail-wm1-f72.google.com (mail-wm1-f72.google.com [209.85.128.72]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-464-WJliWjDBPm2OzsX9a8MIiw-1; Tue, 19 Oct 2021 05:29:15 -0400 Received: by mail-wm1-f72.google.com with SMTP id u14-20020a05600c19ce00b0030d8549d49aso608124wmq.0 for ; Tue, 19 Oct 2021 02:29:15 -0700 (PDT) Received: from localhost (static-233-86-86-188.ipcom.comunitel.net. [188.86.86.233]) by smtp.gmail.com with ESMTPSA id c185sm1749652wma.8.2021.10.19.02.29.13 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 19 Oct 2021 02:29:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1634635756; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=xiin61qQHRQJuFfJtZIvj/8PDAv1mtm9Lzc4rxWOruY=; b=dRlJmDo6G97noKFGlkvWBzG3xsN7I4+UE9BQa76DdoIF0+uxZkAOc3825uDkcWLIN2WUoR JgR+vqSoIsl2P2i2Ioq/36UQ4h4F7wu68L5l8HrEIvlMfk05qElq+4o11HgGlB9alFOndJ n19/UcBWEhtZZbKCzzuVugbY3KQC9Dw= X-MC-Unique: WJliWjDBPm2OzsX9a8MIiw-1 X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=xiin61qQHRQJuFfJtZIvj/8PDAv1mtm9Lzc4rxWOruY=; b=hnRW3DN4IQMY3kJoPbgZw9m1rWrkqiJ1n+lIU5vzPo2mFpbAMpe1rJYrGACO2eqz4U AghaPr4+7JZlYwEzV2dnsifjUq+gr1X3HwD0pS+Aa7t7aGIGT2GdliX3B3janoo1gPzc 8jfYRNsDtPf+2kvp87Ti3eNzz0oQVscrHPdx03Mg0myEb6maJk+l0G3qo2UWh2TSp2v0 42JZhu+BkhdxaIX1VFRJTU8T2wwLb0B+oCXu5YKgkyrm+fEGF2q9xq4wDLDa4mQTWP95 fC3qSFGwiap7PvWQKlt8FY0to5Iyjx8Gc1g34eSlmCmkj0/W5kqfdMIO6oeBiZkoRpV7 RP4g== X-Gm-Message-State: AOAM533BeNGzd+tyjyWAS1eEzY0YQiM/NIzGV1iePL8eJNMhlSku14Sd k3lqiN2IgH1NYz0xLPcGJoFaoxJDTCRShw4V4lQhayuHlSmqG1VsKUwMPIjLAhcoOaUIxUZD2g7 xrRof0gQBfTTwc/E+P08UO3rEaPaxLDX651rN395Cz26UDU3oHkI7ETR+wYKhcxleAAA= X-Received: by 2002:a05:600c:2301:: with SMTP id 1mr4788630wmo.125.1634635754121; Tue, 19 Oct 2021 02:29:14 -0700 (PDT) X-Google-Smtp-Source: ABdhPJydWRELIJrwDPwo8c3E131bNzk6+8/WlO7ZY7+nhTRe/zqlURFJru1WcqDvMBbReNwR5kv1LQ== X-Received: by 2002:a05:600c:2301:: with SMTP id 1mr4788604wmo.125.1634635753910; Tue, 19 Oct 2021 02:29:13 -0700 (PDT) From: Juan Quintela To: qemu-devel@nongnu.org Subject: [PULL 3/7] migration/ram: Don't passs RAMState to migration_clear_memory_region_dirty_bitmap_*() Date: Tue, 19 Oct 2021 11:29:03 +0200 Message-Id: <20211019092907.5255-4-quintela@redhat.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20211019092907.5255-1-quintela@redhat.com> References: <20211019092907.5255-1-quintela@redhat.com> MIME-Version: 1.0 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=quintela@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=quintela@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -16 X-Spam_score: -1.7 X-Spam_bar: - X-Spam_report: (-1.7 / 5.0 requ) BAYES_00=-1.9, DKIM_INVALID=0.1, DKIM_SIGNED=0.1, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=no autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: David Hildenbrand , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , "Dr. David Alan Gilbert" , Peter Xu , Juan Quintela Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZM-MESSAGEID: 1634636265432100001 From: David Hildenbrand The parameter is unused, let's drop it. Reviewed-by: Peter Xu Reviewed-by: Philippe Mathieu-Daud=C3=A9 Reviewed-by: Juan Quintela Signed-off-by: David Hildenbrand Signed-off-by: Juan Quintela --- migration/ram.c | 13 +++++-------- 1 file changed, 5 insertions(+), 8 deletions(-) diff --git a/migration/ram.c b/migration/ram.c index 7a43bfd7af..bb908822d5 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -789,8 +789,7 @@ unsigned long migration_bitmap_find_dirty(RAMState *rs,= RAMBlock *rb, return find_next_bit(bitmap, size, start); } =20 -static void migration_clear_memory_region_dirty_bitmap(RAMState *rs, - RAMBlock *rb, +static void migration_clear_memory_region_dirty_bitmap(RAMBlock *rb, unsigned long page) { uint8_t shift; @@ -818,8 +817,7 @@ static void migration_clear_memory_region_dirty_bitmap(= RAMState *rs, } =20 static void -migration_clear_memory_region_dirty_bitmap_range(RAMState *rs, - RAMBlock *rb, +migration_clear_memory_region_dirty_bitmap_range(RAMBlock *rb, unsigned long start, unsigned long npages) { @@ -832,7 +830,7 @@ migration_clear_memory_region_dirty_bitmap_range(RAMSta= te *rs, * exclusive. */ for (i =3D chunk_start; i < chunk_end; i +=3D chunk_pages) { - migration_clear_memory_region_dirty_bitmap(rs, rb, i); + migration_clear_memory_region_dirty_bitmap(rb, i); } } =20 @@ -850,7 +848,7 @@ static inline bool migration_bitmap_clear_dirty(RAMStat= e *rs, * the page in the chunk we clear the remote dirty bitmap for all. * Clearing it earlier won't be a problem, but too late will. */ - migration_clear_memory_region_dirty_bitmap(rs, rb, page); + migration_clear_memory_region_dirty_bitmap(rb, page); =20 ret =3D test_and_clear_bit(page, rb->bmap); if (ret) { @@ -2777,8 +2775,7 @@ void qemu_guest_free_page_hint(void *addr, size_t len) * are initially set. Otherwise those skipped pages will be sent in * the next round after syncing from the memory region bitmap. */ - migration_clear_memory_region_dirty_bitmap_range(ram_state, block, - start, npages); + migration_clear_memory_region_dirty_bitmap_range(block, start, npa= ges); ram_state->migration_dirty_pages -=3D bitmap_count_one_with_offset(block->bmap, start, npa= ges); bitmap_clear(block->bmap, start, npages); --=20 2.31.1 From nobody Mon May 20 23:52:52 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1634635942037333.7777482339783; Tue, 19 Oct 2021 02:32:22 -0700 (PDT) Received: from localhost ([::1]:42664 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mclTo-0008Dt-Vn for importer@patchew.org; Tue, 19 Oct 2021 05:32:21 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:54758) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mclQv-0003mQ-GM for qemu-devel@nongnu.org; Tue, 19 Oct 2021 05:29:21 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]:37856) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mclQt-0005fl-1v for qemu-devel@nongnu.org; Tue, 19 Oct 2021 05:29:21 -0400 Received: from mail-wm1-f72.google.com (mail-wm1-f72.google.com [209.85.128.72]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-125-305e8nIqOfSmWcsKRSdNIg-1; Tue, 19 Oct 2021 05:29:17 -0400 Received: by mail-wm1-f72.google.com with SMTP id s25-20020a7bc399000000b0030da0f36afeso918258wmj.1 for ; Tue, 19 Oct 2021 02:29:17 -0700 (PDT) Received: from localhost (static-233-86-86-188.ipcom.comunitel.net. [188.86.86.233]) by smtp.gmail.com with ESMTPSA id n11sm14024588wrw.43.2021.10.19.02.29.14 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 19 Oct 2021 02:29:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1634635758; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=sAKgpEl8rTyJC3oyhCxTrd4U/7tfuhWfKE44KangE/I=; b=UVDPbrWzYDQtHeEfylWuHIY+7kUAtKSt1Ge6O2j9I4HHsICU0iwDNrqm9B5/9YUUlTnSRe vWcKesmv+ySSfCrTWV4z0NbJOYsqUJmM/CgVdh5hhqKqf5S77Zy6lzBrEmVoBt2hoqOKiK +ezvCmTFwFqc1Hn1flvYcbwxR25FmVs= X-MC-Unique: 305e8nIqOfSmWcsKRSdNIg-1 X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=sAKgpEl8rTyJC3oyhCxTrd4U/7tfuhWfKE44KangE/I=; b=1prKmsfrjgCpm5CxCg1wJ0Q2teNwRihhwX+/EgX+54oHJRpT0QsDECAv5V1Mtk2Gpa iHQ4Y07CQVe1BZFgZaFNZ+AFhaev26RdPkilS1zleltERkVxiGU5Iy64GwvNdL2+btj1 r1sWKlXtypHuCMHFhZ2SpXhw2syZZW44bLcKPFr0AmznfQqRJtP/dQltzgeQCUNS1woi JzYgKZz1sZi2AGbZdDF3B+qQuCIco79SRmjIj/0uSJHOzK4qFGV4iiGuy2F5gMIrTZz2 7Kr0MxOTlWAQnJIGdwAVgZSzynjzc/FuQrfb4yL9wxKD4tkr//7eLUu9Oh7DMBZMT9ZG HvSg== X-Gm-Message-State: AOAM531hVzZ8W9YBZz/+gBXHvGktXXvh9JNqxtof6z7f2ZbeXxbRJ2bz lUjfqZk+I+XDE6A9JVMG3X6gqoTTHaqzWG97GYaUab50f/yNifQLMOYw340puuKIHwMXuw4tjuK sv899ZPO/E/VTViz1tc7ILNCfw3nQQDfWCVTx61K8qrGIu4VwUWO5GUdRGAh5P1j3Pss= X-Received: by 2002:adf:aad7:: with SMTP id i23mr41663081wrc.209.1634635755684; Tue, 19 Oct 2021 02:29:15 -0700 (PDT) X-Google-Smtp-Source: ABdhPJw9BsDRokjXgWfBSN0ta4Wcv2/CInOW/YYBtUSF1EmPzNiB1Ub8XXupNX+4SCSBv1qzD2r/Rg== X-Received: by 2002:adf:aad7:: with SMTP id i23mr41663053wrc.209.1634635755389; Tue, 19 Oct 2021 02:29:15 -0700 (PDT) From: Juan Quintela To: qemu-devel@nongnu.org Subject: [PULL 4/7] migration: allow multifd for socket protocol only Date: Tue, 19 Oct 2021 11:29:04 +0200 Message-Id: <20211019092907.5255-5-quintela@redhat.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20211019092907.5255-1-quintela@redhat.com> References: <20211019092907.5255-1-quintela@redhat.com> MIME-Version: 1.0 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=quintela@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=quintela@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.049, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: "Dr. David Alan Gilbert" , Li Zhijian , Juan Quintela Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZM-MESSAGEID: 1634635942763100001 Content-Type: text/plain; charset="utf-8" From: Li Zhijian To: , , CC: Li Zhijian Date: Sat, 31 Jul 2021 22:05:51 +0800 (5 weeks, 4 days, 17 hours ago) multifd with unsupported protocol will cause a segment fault. (gdb) bt #0 0x0000563b4a93faf8 in socket_connect (addr=3D0x0, errp=3D0x7f7f0267541= 0) at ../util/qemu-sockets.c:1190 #1 0x0000563b4a797a03 in qio_channel_socket_connect_sync (ioc=3D0x563b4d16e8c0, addr=3D0x0, errp=3D0x7f7f02675410) at ../io/channel-socket.c:145 #2 0x0000563b4a797abf in qio_channel_socket_connect_worker (task=3D0x563b= 4cd86c30, opaque=3D0x0) at ../io/channel-socket.c:168 #3 0x0000563b4a792631 in qio_task_thread_worker (opaque=3D0x563b4cd86c30)= at ../io/task.c:124 #4 0x0000563b4a91da69 in qemu_thread_start (args=3D0x563b4c44bb80) at ../= util/qemu-thread-posix.c:541 #5 0x00007f7fe9b5b3f9 in ?? () #6 0x0000000000000000 in ?? () It's enough to check migrate_multifd_is_allowed() in multifd cleanup() and multifd setup() though there are so many other places using migrate_use_mul= tifd(). Signed-off-by: Li Zhijian Reviewed-by: Juan Quintela Signed-off-by: Juan Quintela --- migration/multifd.h | 2 ++ migration/migration.c | 4 ++++ migration/multifd.c | 24 ++++++++++++++++++++++-- 3 files changed, 28 insertions(+), 2 deletions(-) diff --git a/migration/multifd.h b/migration/multifd.h index 16c4d112d1..15c50ca0b2 100644 --- a/migration/multifd.h +++ b/migration/multifd.h @@ -13,6 +13,8 @@ #ifndef QEMU_MIGRATION_MULTIFD_H #define QEMU_MIGRATION_MULTIFD_H =20 +bool migrate_multifd_is_allowed(void); +void migrate_protocol_allow_multifd(bool allow); int multifd_save_setup(Error **errp); void multifd_save_cleanup(void); int multifd_load_setup(Error **errp); diff --git a/migration/migration.c b/migration/migration.c index 6ac807ef3d..f13b07c638 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -453,10 +453,12 @@ static void qemu_start_incoming_migration(const char = *uri, Error **errp) { const char *p =3D NULL; =20 + migrate_protocol_allow_multifd(false); /* reset it anyway */ qapi_event_send_migration(MIGRATION_STATUS_SETUP); if (strstart(uri, "tcp:", &p) || strstart(uri, "unix:", NULL) || strstart(uri, "vsock:", NULL)) { + migrate_protocol_allow_multifd(true); socket_start_incoming_migration(p ? p : uri, errp); #ifdef CONFIG_RDMA } else if (strstart(uri, "rdma:", &p)) { @@ -2280,9 +2282,11 @@ void qmp_migrate(const char *uri, bool has_blk, bool= blk, } } =20 + migrate_protocol_allow_multifd(false); if (strstart(uri, "tcp:", &p) || strstart(uri, "unix:", NULL) || strstart(uri, "vsock:", NULL)) { + migrate_protocol_allow_multifd(true); socket_start_outgoing_migration(s, p ? p : uri, &local_err); #ifdef CONFIG_RDMA } else if (strstart(uri, "rdma:", &p)) { diff --git a/migration/multifd.c b/migration/multifd.c index efd424bc97..283f672bf0 100644 --- a/migration/multifd.c +++ b/migration/multifd.c @@ -531,7 +531,7 @@ void multifd_save_cleanup(void) { int i; =20 - if (!migrate_use_multifd()) { + if (!migrate_use_multifd() || !migrate_multifd_is_allowed()) { return; } multifd_send_terminate_threads(NULL); @@ -868,6 +868,17 @@ cleanup: multifd_new_send_channel_cleanup(p, sioc, local_err); } =20 +static bool migrate_allow_multifd; +void migrate_protocol_allow_multifd(bool allow) +{ + migrate_allow_multifd =3D allow; +} + +bool migrate_multifd_is_allowed(void) +{ + return migrate_allow_multifd; +} + int multifd_save_setup(Error **errp) { int thread_count; @@ -878,6 +889,11 @@ int multifd_save_setup(Error **errp) if (!migrate_use_multifd()) { return 0; } + if (!migrate_multifd_is_allowed()) { + error_setg(errp, "multifd is not supported by current protocol"); + return -1; + } + s =3D migrate_get_current(); thread_count =3D migrate_multifd_channels(); multifd_send_state =3D g_malloc0(sizeof(*multifd_send_state)); @@ -971,7 +987,7 @@ int multifd_load_cleanup(Error **errp) { int i; =20 - if (!migrate_use_multifd()) { + if (!migrate_use_multifd() || !migrate_multifd_is_allowed()) { return 0; } multifd_recv_terminate_threads(NULL); @@ -1120,6 +1136,10 @@ int multifd_load_setup(Error **errp) if (!migrate_use_multifd()) { return 0; } + if (!migrate_multifd_is_allowed()) { + error_setg(errp, "multifd is not supported by current protocol"); + return -1; + } thread_count =3D migrate_multifd_channels(); multifd_recv_state =3D g_malloc0(sizeof(*multifd_recv_state)); multifd_recv_state->params =3D g_new0(MultiFDRecvParams, thread_count); --=20 2.31.1 From nobody Mon May 20 23:52:52 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1634636229507301.6241178968203; Tue, 19 Oct 2021 02:37:09 -0700 (PDT) Received: from localhost ([::1]:53194 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mclYS-00076h-5l for importer@patchew.org; Tue, 19 Oct 2021 05:37:08 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:54794) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mclQx-0003rr-5j for qemu-devel@nongnu.org; Tue, 19 Oct 2021 05:29:23 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:45393) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mclQu-0005gf-CM for qemu-devel@nongnu.org; Tue, 19 Oct 2021 05:29:22 -0400 Received: from mail-wr1-f72.google.com (mail-wr1-f72.google.com [209.85.221.72]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-195-jrbas4acO26eja1ImanLog-1; Tue, 19 Oct 2021 05:29:18 -0400 Received: by mail-wr1-f72.google.com with SMTP id f1-20020a5d64c1000000b001611832aefeso9927897wri.17 for ; Tue, 19 Oct 2021 02:29:18 -0700 (PDT) Received: from localhost (static-233-86-86-188.ipcom.comunitel.net. [188.86.86.233]) by smtp.gmail.com with ESMTPSA id a16sm14535905wrs.30.2021.10.19.02.29.15 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 19 Oct 2021 02:29:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1634635759; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=CnYbXqplDYjVx6yRtdKoJVb5ZarXQXE39HCFPLbwh08=; b=DoySnws2+rprFJo4IEDBDgo0t529t3jr5z4Le0K8UyPsx/QPjiPJyIJPv20Hw56RTQaJJV 2XqHRRifQdFl+Ee/u43DW2bi5nfRAswsitBtbwUwrmJPxxGlYk7CfReSt4f0plTntvSeNK Yw4BfcOFgrfvbX4cUyYjTkivLXo12I8= X-MC-Unique: jrbas4acO26eja1ImanLog-1 X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=CnYbXqplDYjVx6yRtdKoJVb5ZarXQXE39HCFPLbwh08=; b=e56cXSiQ0sbRZDSyUhlQ/JMbUTJj+qvxmryueTiqplzb2i8cBVi77nzc285sF8ObSK 1oW2NkdTicmbhF/mAXuZdhZPQVUz3dQfTcAkSMWSFLiwnuAckWzM94WyEdy6ZXwDuGIk MXzvdzpGyU7WARRt6uzxsHVEQzZgTzas1mPGwK+3nkNc0ykwTZMKIGaXyvxflQgyUv6A XFeyt0p6wiTsgfk+PTjwNRDsnvBdomjKbSZmNBhA++xlIOANZaMO+ILS3wcnbYLo4h6D Gl5q16W/tRX8ALmxU/ek+yE2nJdUsbi0Jm0o1xUzhKCQJNWCO8VA0eAyssuH8mGIsjQo NAUg== X-Gm-Message-State: AOAM530SomInWgfZ9rgs7HnMn8D/vuGkAPtl4iUsHVUqTwiIBc4LAUBb kpHbYbkZmy4nG4lybgPYQyYU4Sny8Gc4ZnODjvs8WcQxH0m1cbfSnaJh2U0mZAh8F8Gom0JbXza P4nNO+h1SpG+AMQXJJChlTQJX5xkvFHi4hVcTdxZhQyAxzRHWLBDYSuHp/lxHkvzNOpg= X-Received: by 2002:a7b:cd15:: with SMTP id f21mr4802498wmj.76.1634635757175; Tue, 19 Oct 2021 02:29:17 -0700 (PDT) X-Google-Smtp-Source: ABdhPJzi173m8zP0/JWh9prgDMNyOB/wZioFfpobQij7Qz2nyOpWiijbNaV301gA4QGFKpYV8Cc+8w== X-Received: by 2002:a7b:cd15:: with SMTP id f21mr4802468wmj.76.1634635756970; Tue, 19 Oct 2021 02:29:16 -0700 (PDT) From: Juan Quintela To: qemu-devel@nongnu.org Subject: [PULL 5/7] migration: allow enabling mutilfd for specific protocol only Date: Tue, 19 Oct 2021 11:29:05 +0200 Message-Id: <20211019092907.5255-6-quintela@redhat.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20211019092907.5255-1-quintela@redhat.com> References: <20211019092907.5255-1-quintela@redhat.com> MIME-Version: 1.0 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=quintela@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=216.205.24.124; envelope-from=quintela@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -28 X-Spam_score: -2.9 X-Spam_bar: -- X-Spam_report: (-2.9 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.049, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: "Dr. David Alan Gilbert" , Li Zhijian , Juan Quintela Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZM-MESSAGEID: 1634636230857100001 Content-Type: text/plain; charset="utf-8" From: Li Zhijian To: , , CC: Li Zhijian Date: Sat, 31 Jul 2021 22:05:52 +0800 (5 weeks, 4 days, 17 hours ago) And change the default to true so that in '-incoming defer' case, user is a= ble to change multifd capability. Signed-off-by: Li Zhijian Reviewed-by: Juan Quintela Signed-off-by: Juan Quintela --- migration/migration.c | 8 ++++++++ migration/multifd.c | 2 +- 2 files changed, 9 insertions(+), 1 deletion(-) diff --git a/migration/migration.c b/migration/migration.c index f13b07c638..9172686b89 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -1237,6 +1237,14 @@ static bool migrate_caps_check(bool *cap_list, } } =20 + /* incoming side only */ + if (runstate_check(RUN_STATE_INMIGRATE) && + !migrate_multifd_is_allowed() && + cap_list[MIGRATION_CAPABILITY_MULTIFD]) { + error_setg(errp, "multifd is not supported by current protocol"); + return false; + } + return true; } =20 diff --git a/migration/multifd.c b/migration/multifd.c index 283f672bf0..7c9deb1921 100644 --- a/migration/multifd.c +++ b/migration/multifd.c @@ -868,7 +868,7 @@ cleanup: multifd_new_send_channel_cleanup(p, sioc, local_err); } =20 -static bool migrate_allow_multifd; +static bool migrate_allow_multifd =3D true; void migrate_protocol_allow_multifd(bool allow) { migrate_allow_multifd =3D allow; --=20 2.31.1 From nobody Mon May 20 23:52:52 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1634636171626474.0359834063552; Tue, 19 Oct 2021 02:36:11 -0700 (PDT) Received: from localhost ([::1]:50652 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mclXW-0005Ic-MQ for importer@patchew.org; Tue, 19 Oct 2021 05:36:10 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:54826) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mclQz-0003zs-OG for qemu-devel@nongnu.org; Tue, 19 Oct 2021 05:29:25 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:41969) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mclQx-0005z8-Mz for qemu-devel@nongnu.org; Tue, 19 Oct 2021 05:29:25 -0400 Received: from mail-wm1-f71.google.com (mail-wm1-f71.google.com [209.85.128.71]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-276-Y3fN5ZR6NHOUFbtKwqcTuQ-1; Tue, 19 Oct 2021 05:29:19 -0400 Received: by mail-wm1-f71.google.com with SMTP id s25-20020a7bc399000000b0030da0f36afeso918325wmj.1 for ; Tue, 19 Oct 2021 02:29:19 -0700 (PDT) Received: from localhost (static-233-86-86-188.ipcom.comunitel.net. [188.86.86.233]) by smtp.gmail.com with ESMTPSA id r4sm18320888wrz.58.2021.10.19.02.29.17 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 19 Oct 2021 02:29:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1634635763; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=pdpP0Zp1A+q+fYKgjolSVvB7+MurDG7TM3Ho1yQv9xE=; b=jNs3biwZmj0oPZSpOekvrJqZ4E0cvu09dCXairJhREcdGMgMl3F1Ehb2031C8tWi26ubdK 0dzk/g/dQA4RvNpxIcaccfjPIc0Q8OT0XkHc+OQLi41y4+WHerTKbww14a94MB8j7fkQM7 l3ptMZwY8jXdHDsQnAGAfjuRsz9iyog= X-MC-Unique: Y3fN5ZR6NHOUFbtKwqcTuQ-1 X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=pdpP0Zp1A+q+fYKgjolSVvB7+MurDG7TM3Ho1yQv9xE=; b=QD2/flYXpuk1hk+adSFizqIMV26IqMXT3fX663nYF+pbp2BROBPclpFMZ18sRd/nL4 bWhtFB9+7JawzMg43oz6HeZANeKDtogUEAVFV42tWzu2mSwMGKdCQ9aZe5BtXoCh7YKR lxoYxsNI5BemFfjIvxYo/3SOsWbOUTOdO1gyqcSIC1rmCuVPOxuIiysoRe5eFt1TeYmR XByMhsbYED5tEYNnVGoPP3sCZqAEYS+fUdJ7K6pLdZ8YjkOpfAYSvhgOoOe4u+tUkdSJ 3LSoyDlPbncNM6osjsG/xbtdGmxFSgOQ/PTKYC8U1aWZiY0N51iBj/tTb8JrjbE1IFg7 vNgg== X-Gm-Message-State: AOAM530rUydlbYqk7q3TdPGFIsCsZG4G1jxUQOMSZZKidDwAaEcwiocT SRwex2N8XXT3FlAz27TUKrvZa9TCgFm7WTbND4Iamw5adOPhtxb2ZxS75wphObqa+YN2aCvguc4 IJGVkhBDNYMfJu6gVQb8r9Tk4aOStwlp1s3Z+f5TjZuzJjZ5fS6ISp0mFs+w3CvnBsOg= X-Received: by 2002:a05:6000:1869:: with SMTP id d9mr41543640wri.161.1634635758236; Tue, 19 Oct 2021 02:29:18 -0700 (PDT) X-Google-Smtp-Source: ABdhPJzPaCka0D7C1Ihf8BbB+Y34jq73qxUJvJelJCY+iz4BJTwthAOT+OHIOXB8tVRSdzkrJ2Vhrg== X-Received: by 2002:a05:6000:1869:: with SMTP id d9mr41543615wri.161.1634635758010; Tue, 19 Oct 2021 02:29:18 -0700 (PDT) From: Juan Quintela To: qemu-devel@nongnu.org Subject: [PULL 6/7] migration/rdma: Try to register On-Demand Paging memory region Date: Tue, 19 Oct 2021 11:29:06 +0200 Message-Id: <20211019092907.5255-7-quintela@redhat.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20211019092907.5255-1-quintela@redhat.com> References: <20211019092907.5255-1-quintela@redhat.com> MIME-Version: 1.0 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=quintela@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=quintela@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -28 X-Spam_score: -2.9 X-Spam_bar: -- X-Spam_report: (-2.9 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.049, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: "Dr. David Alan Gilbert" , Li Zhijian , Juan Quintela Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZM-MESSAGEID: 1634636173292100001 Content-Type: text/plain; charset="utf-8" From: Li Zhijian Previously, for the fsdax mem-backend-file, it will register failed with Operation not supported. In this case, we can try to register it with On-Demand Paging[1] like what rpma_mr_reg() does on rpma[2]. [1]: https://community.mellanox.com/s/article/understanding-on-demand-pagin= g--odp-x [2]: http://pmem.io/rpma/manpages/v0.9.0/rpma_mr_reg.3 CC: Marcel Apfelbaum Signed-off-by: Li Zhijian Reviewed-by: Marcel Apfelbaum Reviewed-by: Juan Quintela Signed-off-by: Juan Quintela --- migration/rdma.c | 73 ++++++++++++++++++++++++++++++------------ migration/trace-events | 1 + 2 files changed, 54 insertions(+), 20 deletions(-) diff --git a/migration/rdma.c b/migration/rdma.c index 5c2d113aa9..eb80431aae 100644 --- a/migration/rdma.c +++ b/migration/rdma.c @@ -1117,19 +1117,47 @@ static int qemu_rdma_alloc_qp(RDMAContext *rdma) return 0; } =20 +/* Check whether On-Demand Paging is supported by RDAM device */ +static bool rdma_support_odp(struct ibv_context *dev) +{ + struct ibv_device_attr_ex attr =3D {0}; + int ret =3D ibv_query_device_ex(dev, NULL, &attr); + if (ret) { + return false; + } + + if (attr.odp_caps.general_caps & IBV_ODP_SUPPORT) { + return true; + } + + return false; +} + static int qemu_rdma_reg_whole_ram_blocks(RDMAContext *rdma) { int i; RDMALocalBlocks *local =3D &rdma->local_ram_blocks; =20 for (i =3D 0; i < local->nb_blocks; i++) { + int access =3D IBV_ACCESS_LOCAL_WRITE | IBV_ACCESS_REMOTE_WRITE; + local->block[i].mr =3D ibv_reg_mr(rdma->pd, local->block[i].local_host_addr, - local->block[i].length, - IBV_ACCESS_LOCAL_WRITE | - IBV_ACCESS_REMOTE_WRITE + local->block[i].length, access ); + + if (!local->block[i].mr && + errno =3D=3D ENOTSUP && rdma_support_odp(rdma->verbs)) { + access |=3D IBV_ACCESS_ON_DEMAND; + /* register ODP mr */ + local->block[i].mr =3D + ibv_reg_mr(rdma->pd, + local->block[i].local_host_addr, + local->block[i].length, access); + trace_qemu_rdma_register_odp_mr(local->block[i].block_name= ); + } + if (!local->block[i].mr) { perror("Failed to register local dest ram block!"); break; @@ -1215,28 +1243,33 @@ static int qemu_rdma_register_and_get_keys(RDMACont= ext *rdma, */ if (!block->pmr[chunk]) { uint64_t len =3D chunk_end - chunk_start; + int access =3D rkey ? IBV_ACCESS_LOCAL_WRITE | IBV_ACCESS_REMOTE_W= RITE : + 0; =20 trace_qemu_rdma_register_and_get_keys(len, chunk_start); =20 - block->pmr[chunk] =3D ibv_reg_mr(rdma->pd, - chunk_start, len, - (rkey ? (IBV_ACCESS_LOCAL_WRITE | - IBV_ACCESS_REMOTE_WRITE) : 0)); - - if (!block->pmr[chunk]) { - perror("Failed to register chunk!"); - fprintf(stderr, "Chunk details: block: %d chunk index %d" - " start %" PRIuPTR " end %" PRIuPTR - " host %" PRIuPTR - " local %" PRIuPTR " registrations: %d\n", - block->index, chunk, (uintptr_t)chunk_start, - (uintptr_t)chunk_end, host_addr, - (uintptr_t)block->local_host_addr, - rdma->total_registrations); - return -1; + block->pmr[chunk] =3D ibv_reg_mr(rdma->pd, chunk_start, len, acces= s); + if (!block->pmr[chunk] && + errno =3D=3D ENOTSUP && rdma_support_odp(rdma->verbs)) { + access |=3D IBV_ACCESS_ON_DEMAND; + /* register ODP mr */ + block->pmr[chunk] =3D ibv_reg_mr(rdma->pd, chunk_start, len, a= ccess); + trace_qemu_rdma_register_odp_mr(block->block_name); } - rdma->total_registrations++; } + if (!block->pmr[chunk]) { + perror("Failed to register chunk!"); + fprintf(stderr, "Chunk details: block: %d chunk index %d" + " start %" PRIuPTR " end %" PRIuPTR + " host %" PRIuPTR + " local %" PRIuPTR " registrations: %d\n", + block->index, chunk, (uintptr_t)chunk_start, + (uintptr_t)chunk_end, host_addr, + (uintptr_t)block->local_host_addr, + rdma->total_registrations); + return -1; + } + rdma->total_registrations++; =20 if (lkey) { *lkey =3D block->pmr[chunk]->lkey; diff --git a/migration/trace-events b/migration/trace-events index a1c0f034ab..5f6aa580de 100644 --- a/migration/trace-events +++ b/migration/trace-events @@ -212,6 +212,7 @@ qemu_rdma_poll_write(const char *compstr, int64_t comp,= int left, uint64_t block qemu_rdma_poll_other(const char *compstr, int64_t comp, int left) "other c= ompletion %s (%" PRId64 ") received left %d" qemu_rdma_post_send_control(const char *desc) "CONTROL: sending %s.." qemu_rdma_register_and_get_keys(uint64_t len, void *start) "Registering %"= PRIu64 " bytes @ %p" +qemu_rdma_register_odp_mr(const char *name) "Try to register On-Demand Pag= ing memory region: %s" qemu_rdma_registration_handle_compress(int64_t length, int index, int64_t = offset) "Zapping zero chunk: %" PRId64 " bytes, index %d, offset %" PRId64 qemu_rdma_registration_handle_finished(void) "" qemu_rdma_registration_handle_ram_blocks(void) "" --=20 2.31.1 From nobody Mon May 20 23:52:52 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 163463591382992.56449286844145; Tue, 19 Oct 2021 02:31:53 -0700 (PDT) Received: from localhost ([::1]:40712 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mclTM-0006sG-Qf for importer@patchew.org; Tue, 19 Oct 2021 05:31:52 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:54818) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mclQy-0003wS-Km for qemu-devel@nongnu.org; Tue, 19 Oct 2021 05:29:24 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:59719) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mclQw-0005u4-Rh for qemu-devel@nongnu.org; Tue, 19 Oct 2021 05:29:24 -0400 Received: from mail-wm1-f72.google.com (mail-wm1-f72.google.com [209.85.128.72]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-211-b2ViE165N-6uuT-0fqs70g-1; Tue, 19 Oct 2021 05:29:21 -0400 Received: by mail-wm1-f72.google.com with SMTP id n9-20020a1c7209000000b0030da7d466b8so912559wmc.5 for ; Tue, 19 Oct 2021 02:29:20 -0700 (PDT) Received: from localhost (static-233-86-86-188.ipcom.comunitel.net. [188.86.86.233]) by smtp.gmail.com with ESMTPSA id h206sm1748558wmh.33.2021.10.19.02.29.18 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 19 Oct 2021 02:29:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1634635762; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=apcd1sPYUX4KQiw1ezZjVt1xKXzOXDw++gGzZ5Ff3ns=; b=YJ0PIJYo8RRbmo3wDBMuLqe3I+jtfsgAeUXwUuJHESn9JAc5qGeXn49495QEFDZYVueB3D cd3wqFd68wme/WkUY8GvwVudFNXc+JeQyLcwBtcx3JjE83lkZr7Idcx1J1Fiq/qooTJrJb cQC++G1rp5nYoLeUYKYXdtX++r2Z8vo= X-MC-Unique: b2ViE165N-6uuT-0fqs70g-1 X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=apcd1sPYUX4KQiw1ezZjVt1xKXzOXDw++gGzZ5Ff3ns=; b=lgv7ZNKDTQMOFVK4wrf66T7DPcMRc3ilmeIKJV5pucY8jiD3iDekrv9jT9b5Hh756q rX4M5kbKmkFHkW2HPG70My3QtDzmusLkWCE5gzC9Z6hHpbS6T8becJO8HRZhJ2DUbzjw 7Mip2FNBPFDGNryZXJj7IMYE7lis58qdJoOV040NW2nZEscfEDCV8QL+mFvKHtL5jWLb rGecFUTbpMz2CxwB1IDE+A+KiPxFsWETQCLqXs0vgz4k2XHHM+n88dNieKS8qc5BqfMg 9/6gNynMoWocdiht/nC6e0c6wem3h7t3AJby+8xjDu6nre4S8GRUVYBWbgymgaWUJjss 4GWg== X-Gm-Message-State: AOAM531buYURqQ55JY7QznveskAtm+bsCM1dukJM87cQMnVgxy30DyKo QsB2wJSA1T8lWFVHodeqks9r1wI4fEzIQ3J6NIjl/qFWoSUqrBfggImcowSCuwWHNt7vom7Nppl sz/QfOsGABhKWnibe5okcrGrlsDoND6rJPz1pg8HRvBBIGPuqYkU0xHm2T7lHHOSetjg= X-Received: by 2002:a05:600c:4108:: with SMTP id j8mr4869485wmi.15.1634635759579; Tue, 19 Oct 2021 02:29:19 -0700 (PDT) X-Google-Smtp-Source: ABdhPJwfFAD/8L8nARnSLtvu9bs9wJUj+mgHzjLDeKqNnIpTANK3pwcwiJKwcxIA0E/obBGgsqkxQg== X-Received: by 2002:a05:600c:4108:: with SMTP id j8mr4869451wmi.15.1634635759249; Tue, 19 Oct 2021 02:29:19 -0700 (PDT) From: Juan Quintela To: qemu-devel@nongnu.org Subject: [PULL 7/7] migration/rdma: advise prefetch write for ODP region Date: Tue, 19 Oct 2021 11:29:07 +0200 Message-Id: <20211019092907.5255-8-quintela@redhat.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20211019092907.5255-1-quintela@redhat.com> References: <20211019092907.5255-1-quintela@redhat.com> MIME-Version: 1.0 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=quintela@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=216.205.24.124; envelope-from=quintela@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -28 X-Spam_score: -2.9 X-Spam_bar: -- X-Spam_report: (-2.9 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.049, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: "Dr. David Alan Gilbert" , Li Zhijian , Juan Quintela Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZM-MESSAGEID: 1634635915450100005 Content-Type: text/plain; charset="utf-8" From: Li Zhijian The responder mr registering with ODP will sent RNR NAK back to the requester in the face of the page fault. --------- ibv_poll_cq wc.status=3D13 RNR retry counter exceeded! ibv_poll_cq wrid=3DWRITE RDMA! --------- ibv_advise_mr(3) helps to make pages present before the actual IO is conducted so that the responder does page fault as little as possible. Signed-off-by: Li Zhijian Reviewed-by: Marcel Apfelbaum Reviewed-by: Juan Quintela Signed-off-by: Juan Quintela --- meson.build | 6 ++++++ migration/rdma.c | 42 ++++++++++++++++++++++++++++++++++++++++++ migration/trace-events | 1 + 3 files changed, 49 insertions(+) diff --git a/meson.build b/meson.build index 5e7946776d..9ed9a993e2 100644 --- a/meson.build +++ b/meson.build @@ -1530,6 +1530,12 @@ config_host_data.set('HAVE_COPY_FILE_RANGE', cc.has_= function('copy_file_range')) config_host_data.set('HAVE_OPENPTY', cc.has_function('openpty', dependenci= es: util)) config_host_data.set('HAVE_STRCHRNUL', cc.has_function('strchrnul')) config_host_data.set('HAVE_SYSTEM_FUNCTION', cc.has_function('system', pre= fix: '#include ')) +if rdma.found() + config_host_data.set('HAVE_IBV_ADVISE_MR', + cc.has_function('ibv_advise_mr', + args: config_host['RDMA_LIBS'].spli= t(), + prefix: '#include ')) +endif =20 # has_header_symbol config_host_data.set('CONFIG_BYTESWAP_H', diff --git a/migration/rdma.c b/migration/rdma.c index eb80431aae..2a3c7889b9 100644 --- a/migration/rdma.c +++ b/migration/rdma.c @@ -1133,6 +1133,32 @@ static bool rdma_support_odp(struct ibv_context *dev) return false; } =20 +/* + * ibv_advise_mr to avoid RNR NAK error as far as possible. + * The responder mr registering with ODP will sent RNR NAK back to + * the requester in the face of the page fault. + */ +static void qemu_rdma_advise_prefetch_mr(struct ibv_pd *pd, uint64_t addr, + uint32_t len, uint32_t lkey, + const char *name, bool wr) +{ +#ifdef HAVE_IBV_ADVISE_MR + int ret; + int advice =3D wr ? IBV_ADVISE_MR_ADVICE_PREFETCH_WRITE : + IBV_ADVISE_MR_ADVICE_PREFETCH; + struct ibv_sge sg_list =3D {.lkey =3D lkey, .addr =3D addr, .length = =3D len}; + + ret =3D ibv_advise_mr(pd, advice, + IBV_ADVISE_MR_FLAG_FLUSH, &sg_list, 1); + /* ignore the error */ + if (ret) { + trace_qemu_rdma_advise_mr(name, len, addr, strerror(errno)); + } else { + trace_qemu_rdma_advise_mr(name, len, addr, "successed"); + } +#endif +} + static int qemu_rdma_reg_whole_ram_blocks(RDMAContext *rdma) { int i; @@ -1156,6 +1182,15 @@ static int qemu_rdma_reg_whole_ram_blocks(RDMAContex= t *rdma) local->block[i].local_host_addr, local->block[i].length, access); trace_qemu_rdma_register_odp_mr(local->block[i].block_name= ); + + if (local->block[i].mr) { + qemu_rdma_advise_prefetch_mr(rdma->pd, + (uintptr_t)local->block[i].local_host_= addr, + local->block[i].length, + local->block[i].mr->lkey, + local->block[i].block_name, + true); + } } =20 if (!local->block[i].mr) { @@ -1255,6 +1290,13 @@ static int qemu_rdma_register_and_get_keys(RDMAConte= xt *rdma, /* register ODP mr */ block->pmr[chunk] =3D ibv_reg_mr(rdma->pd, chunk_start, len, a= ccess); trace_qemu_rdma_register_odp_mr(block->block_name); + + if (block->pmr[chunk]) { + qemu_rdma_advise_prefetch_mr(rdma->pd, (uintptr_t)chunk_st= art, + len, block->pmr[chunk]->lkey, + block->block_name, rkey); + + } } } if (!block->pmr[chunk]) { diff --git a/migration/trace-events b/migration/trace-events index 5f6aa580de..a8ae163707 100644 --- a/migration/trace-events +++ b/migration/trace-events @@ -213,6 +213,7 @@ qemu_rdma_poll_other(const char *compstr, int64_t comp,= int left) "other complet qemu_rdma_post_send_control(const char *desc) "CONTROL: sending %s.." qemu_rdma_register_and_get_keys(uint64_t len, void *start) "Registering %"= PRIu64 " bytes @ %p" qemu_rdma_register_odp_mr(const char *name) "Try to register On-Demand Pag= ing memory region: %s" +qemu_rdma_advise_mr(const char *name, uint32_t len, uint64_t addr, const c= har *res) "Try to advise block %s prefetch at %" PRIu32 "@0x%" PRIx64 ": %s" qemu_rdma_registration_handle_compress(int64_t length, int index, int64_t = offset) "Zapping zero chunk: %" PRId64 " bytes, index %d, offset %" PRId64 qemu_rdma_registration_handle_finished(void) "" qemu_rdma_registration_handle_ram_blocks(void) "" --=20 2.31.1