From nobody Mon Feb 9 00:30:16 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 164612415990642.11310471646277; Tue, 1 Mar 2022 00:42:39 -0800 (PST) Received: from localhost ([::1]:39396 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nOy5e-0006FN-Oc for importer@patchew.org; Tue, 01 Mar 2022 03:42:38 -0500 Received: from eggs.gnu.org ([209.51.188.92]:53940) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nOy2o-0003Pn-5A for qemu-devel@nongnu.org; Tue, 01 Mar 2022 03:39:43 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]:26013) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nOy2l-0007qn-Ex for qemu-devel@nongnu.org; Tue, 01 Mar 2022 03:39:40 -0500 Received: from mail-pj1-f70.google.com (mail-pj1-f70.google.com [209.85.216.70]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-161-vPW9ZhuLNky2wui2wmSUVQ-1; Tue, 01 Mar 2022 03:39:37 -0500 Received: by mail-pj1-f70.google.com with SMTP id mn16-20020a17090b189000b001bc2315d75cso1209621pjb.3 for ; Tue, 01 Mar 2022 00:39:37 -0800 (PST) Received: from localhost.localdomain ([94.177.118.144]) by smtp.gmail.com with ESMTPSA id l1-20020a17090aec0100b001bc6d8bb27dsm1439987pjy.37.2022.03.01.00.39.33 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Tue, 01 Mar 2022 00:39:35 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1646123978; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=TVsGHM6RhLibS9NK15rpLaSdALDGeWM7BQJKbM6wKrI=; b=h4MvnHb5ibJmUg5+H6fAwVECn/LddOUYX9A0xCbRb70501lofnMSx8ISxmE8YcpNVUnsHk qaXQQumSKymXbwjkFRtEjbTW57DmWu+C8rRDVk6GlFSSN36oonqtDxFtf0wzzINpWvqHXG qj/CHtrN/UPpqqonh6G1qhNWRaddPgY= X-MC-Unique: vPW9ZhuLNky2wui2wmSUVQ-1 X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=TVsGHM6RhLibS9NK15rpLaSdALDGeWM7BQJKbM6wKrI=; b=bWfywLoJyXbKuC6CFrR7m8nyRRjeW1X9PcIxu0PbOJXcw4nbHUDV401vUtuf5Rmbpu TTbbd1kMvf7cSYoe3kmWqSyRLlcCylBhHyVmrNMV2x2H7eKDeQE5L/euiZsarF+2EpXf qfDZwIZEebDPSQhN4Uk51b0cXI6NG+0uxkGe/c1NAo0PxrgGScxFqy96SY/IavhIwgdw vF+JddLTVJgh0ErYn6nGtFdc+BHtBPjztnqiM0ei2PI6ZmxyFdUtLm1qr9ZjXMdaiiOD HAVnwIyQP5UlJ2eSXRaTjFp6rGUK2OWC09uRk3brxLn7kuVHDBaLwN+wDzbgQn/l211Z AhwQ== X-Gm-Message-State: AOAM532yxVzFuL7/Ho/B0l7eRfrtk3jnVFWWGjcjxKUOAZrCV6M+ZJmX fXKCLNcfTwMviTKel06JaZ6bD567AZ5UkoH58YMtYzYLeP0QUjISD0qNXkE0mg68V6TS5zA5Ltw 8NvvlmiWBhCR8iEh0sIZk0pKkg05Lgik58Qk1jXDPSF54yGNHyURs3pipEAQ4Vxlx X-Received: by 2002:a17:90a:b307:b0:1bd:37f3:f0fc with SMTP id d7-20020a17090ab30700b001bd37f3f0fcmr11501934pjr.132.1646123976440; Tue, 01 Mar 2022 00:39:36 -0800 (PST) X-Google-Smtp-Source: ABdhPJzCHRN6RyLdYV/s8bdOrppKIlqiT6rI4Dhe74fFJPDbM1jFZuliGx+lOdQUERVeQPMr52DDRw== X-Received: by 2002:a17:90a:b307:b0:1bd:37f3:f0fc with SMTP id d7-20020a17090ab30700b001bd37f3f0fcmr11501911pjr.132.1646123975909; Tue, 01 Mar 2022 00:39:35 -0800 (PST) From: Peter Xu To: qemu-devel@nongnu.org Subject: [PATCH v2 01/25] migration: Dump sub-cmd name in loadvm_process_command tp Date: Tue, 1 Mar 2022 16:39:01 +0800 Message-Id: <20220301083925.33483-2-peterx@redhat.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20220301083925.33483-1-peterx@redhat.com> References: <20220301083925.33483-1-peterx@redhat.com> MIME-Version: 1.0 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=peterx@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=peterx@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -28 X-Spam_score: -2.9 X-Spam_bar: -- X-Spam_report: (-2.9 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.082, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H5=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Juan Quintela , "Dr . David Alan Gilbert" , peterx@redhat.com, Leonardo Bras Soares Passos Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZM-MESSAGEID: 1646124162218100001 Content-Type: text/plain; charset="utf-8" It'll be easier to read the name rather than index of sub-cmd when debuggin= g. Reviewed-by: Dr. David Alan Gilbert Signed-off-by: Peter Xu --- migration/savevm.c | 3 ++- migration/trace-events | 2 +- 2 files changed, 3 insertions(+), 2 deletions(-) diff --git a/migration/savevm.c b/migration/savevm.c index 1599b02fbc..7bb65e1d61 100644 --- a/migration/savevm.c +++ b/migration/savevm.c @@ -2273,12 +2273,13 @@ static int loadvm_process_command(QEMUFile *f) return qemu_file_get_error(f); } =20 - trace_loadvm_process_command(cmd, len); if (cmd >=3D MIG_CMD_MAX || cmd =3D=3D MIG_CMD_INVALID) { error_report("MIG_CMD 0x%x unknown (len 0x%x)", cmd, len); return -EINVAL; } =20 + trace_loadvm_process_command(mig_cmd_args[cmd].name, len); + if (mig_cmd_args[cmd].len !=3D -1 && mig_cmd_args[cmd].len !=3D len) { error_report("%s received with bad length - expecting %zu, got %d", mig_cmd_args[cmd].name, diff --git a/migration/trace-events b/migration/trace-events index 48aa7b10ee..123cfe79d7 100644 --- a/migration/trace-events +++ b/migration/trace-events @@ -22,7 +22,7 @@ loadvm_postcopy_handle_resume(void) "" loadvm_postcopy_ram_handle_discard(void) "" loadvm_postcopy_ram_handle_discard_end(void) "" loadvm_postcopy_ram_handle_discard_header(const char *ramid, uint16_t len)= "%s: %ud" -loadvm_process_command(uint16_t com, uint16_t len) "com=3D0x%x len=3D%d" +loadvm_process_command(const char *s, uint16_t len) "com=3D%s len=3D%d" loadvm_process_command_ping(uint32_t val) "0x%x" postcopy_ram_listen_thread_exit(void) "" postcopy_ram_listen_thread_start(void) "" --=20 2.32.0 From nobody Mon Feb 9 00:30:16 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1646124164247964.3835754101775; Tue, 1 Mar 2022 00:42:44 -0800 (PST) Received: from localhost ([::1]:39698 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nOy5j-0006TX-7B for importer@patchew.org; Tue, 01 Mar 2022 03:42:43 -0500 Received: from eggs.gnu.org ([209.51.188.92]:54004) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nOy2s-0003QT-55 for qemu-devel@nongnu.org; Tue, 01 Mar 2022 03:39:47 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:41694) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nOy2p-0007rT-79 for qemu-devel@nongnu.org; Tue, 01 Mar 2022 03:39:44 -0500 Received: from mail-pf1-f197.google.com (mail-pf1-f197.google.com [209.85.210.197]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-448-gWaJIurOOtCPgKzOpdy5Fg-1; Tue, 01 Mar 2022 03:39:40 -0500 Received: by mail-pf1-f197.google.com with SMTP id t184-20020a6281c1000000b004e103c5f726so880850pfd.8 for ; Tue, 01 Mar 2022 00:39:40 -0800 (PST) Received: from localhost.localdomain ([94.177.118.144]) by smtp.gmail.com with ESMTPSA id l1-20020a17090aec0100b001bc6d8bb27dsm1439987pjy.37.2022.03.01.00.39.36 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Tue, 01 Mar 2022 00:39:38 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1646123981; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=0fllUttO/wJ4RRPewl/b4FdObxc5ST7EKpTLC+RNJ4I=; b=YnvoegB6yllM6D3CfEeeCiAwWLV+OCsDgSHfLKLlwkYJsZaCh1nq5diGnIqTPeVYPcZPBO MjGskTMgYg0Uo1AwDVdspAu/4uiF+rhv4KPLbGKtVTDOOGvhvWUTegQtxg31cT/WFVo33u RmlIjecwWJSIUH17jEMlJRxNt6ub+vU= X-MC-Unique: gWaJIurOOtCPgKzOpdy5Fg-1 X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=0fllUttO/wJ4RRPewl/b4FdObxc5ST7EKpTLC+RNJ4I=; b=f53WlxQ/+SHUWJCj58YmQ9IBzbqNoZYdiiCankDp8D6a90By9dWIYcO6Q+4gwQfyQl spmDZTVTsuQ4RabR5aZdhWXge5U/7QR998an6LCiR/h+TaF8H76PrFOXron0v0r9HDWg pc6HXQcajKP7d414fET8hOBcfEcR7JKBqQjL5GC8c41ZoRZLM8q7ullLA6NPQlwfXesv 2zUhBMmoLvMPEerbMShBeYi9U9ZwUjk8kQgpn16Wr2hdv72huOjAylwU5ekj/SO/MoCV utHYEAYrXXL4WUYqENU4ymT56P5To1Z8NwBteggD2ZL+/RxMb7ss7z9/4k6+k8K4zwPh osvQ== X-Gm-Message-State: AOAM530t7Ky9eX+v8FZ/mR6u+5PjP3+ZhwqyuPJO4fTw24IoqsK9qGJF rh2Yos2o8r1+mDz+m/ShNhjWI3msHBg0P9X/SM7WHxT2vFombJTj2ywqFLwTb+FBd4WpF/Qxcaf CSVidFVvFUDJlc0BA22nGcQyFKuUTuWng1Hhmg84eB2srYQg7auWTZ4uMesMUoQ4X X-Received: by 2002:a17:902:ccca:b0:150:406:6236 with SMTP id z10-20020a170902ccca00b0015004066236mr24732461ple.67.1646123979276; Tue, 01 Mar 2022 00:39:39 -0800 (PST) X-Google-Smtp-Source: ABdhPJyvOklBL3FPFWExxD+jwXY5ryVCuHSg7F28gu7zBhUiKmi7DQSJf/OAXkr209UnfZD4mb4rRQ== X-Received: by 2002:a17:902:ccca:b0:150:406:6236 with SMTP id z10-20020a170902ccca00b0015004066236mr24732444ple.67.1646123978874; Tue, 01 Mar 2022 00:39:38 -0800 (PST) From: Peter Xu To: qemu-devel@nongnu.org Subject: [PATCH v2 02/25] migration: Finer grained tracepoints for POSTCOPY_LISTEN Date: Tue, 1 Mar 2022 16:39:02 +0800 Message-Id: <20220301083925.33483-3-peterx@redhat.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20220301083925.33483-1-peterx@redhat.com> References: <20220301083925.33483-1-peterx@redhat.com> MIME-Version: 1.0 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=peterx@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=peterx@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -28 X-Spam_score: -2.9 X-Spam_bar: -- X-Spam_report: (-2.9 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.082, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H5=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Juan Quintela , "Dr . David Alan Gilbert" , peterx@redhat.com, Leonardo Bras Soares Passos Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZM-MESSAGEID: 1646124165433100007 Content-Type: text/plain; charset="utf-8" The enablement of postcopy listening has a few steps, add a few tracepoints= to be there ready for some basic measurements for them. Reviewed-by: Dr. David Alan Gilbert Signed-off-by: Peter Xu --- migration/savevm.c | 9 ++++++++- migration/trace-events | 2 +- 2 files changed, 9 insertions(+), 2 deletions(-) diff --git a/migration/savevm.c b/migration/savevm.c index 7bb65e1d61..190cc5fc42 100644 --- a/migration/savevm.c +++ b/migration/savevm.c @@ -1948,9 +1948,10 @@ static void *postcopy_ram_listen_thread(void *opaque) static int loadvm_postcopy_handle_listen(MigrationIncomingState *mis) { PostcopyState ps =3D postcopy_state_set(POSTCOPY_INCOMING_LISTENING); - trace_loadvm_postcopy_handle_listen(); Error *local_err =3D NULL; =20 + trace_loadvm_postcopy_handle_listen("enter"); + if (ps !=3D POSTCOPY_INCOMING_ADVISE && ps !=3D POSTCOPY_INCOMING_DISC= ARD) { error_report("CMD_POSTCOPY_LISTEN in wrong postcopy state (%d)", p= s); return -1; @@ -1965,6 +1966,8 @@ static int loadvm_postcopy_handle_listen(MigrationInc= omingState *mis) } } =20 + trace_loadvm_postcopy_handle_listen("after discard"); + /* * Sensitise RAM - can now generate requests for blocks that don't exi= st * However, at this point the CPU shouldn't be running, and the IO @@ -1977,6 +1980,8 @@ static int loadvm_postcopy_handle_listen(MigrationInc= omingState *mis) } } =20 + trace_loadvm_postcopy_handle_listen("after uffd"); + if (postcopy_notify(POSTCOPY_NOTIFY_INBOUND_LISTEN, &local_err)) { error_report_err(local_err); return -1; @@ -1991,6 +1996,8 @@ static int loadvm_postcopy_handle_listen(MigrationInc= omingState *mis) qemu_sem_wait(&mis->listen_thread_sem); qemu_sem_destroy(&mis->listen_thread_sem); =20 + trace_loadvm_postcopy_handle_listen("return"); + return 0; } =20 diff --git a/migration/trace-events b/migration/trace-events index 123cfe79d7..92596c00d8 100644 --- a/migration/trace-events +++ b/migration/trace-events @@ -14,7 +14,7 @@ loadvm_handle_cmd_packaged_main(int ret) "%d" loadvm_handle_cmd_packaged_received(int ret) "%d" loadvm_handle_recv_bitmap(char *s) "%s" loadvm_postcopy_handle_advise(void) "" -loadvm_postcopy_handle_listen(void) "" +loadvm_postcopy_handle_listen(const char *str) "%s" loadvm_postcopy_handle_run(void) "" loadvm_postcopy_handle_run_cpu_sync(void) "" loadvm_postcopy_handle_run_vmstart(void) "" --=20 2.32.0 From nobody Mon Feb 9 00:30:16 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1646124599995828.632995090107; Tue, 1 Mar 2022 00:49:59 -0800 (PST) Received: from localhost ([::1]:47948 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nOyCk-0003zn-G9 for importer@patchew.org; Tue, 01 Mar 2022 03:49:58 -0500 Received: from eggs.gnu.org ([209.51.188.92]:54070) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nOy2t-0003RG-93 for qemu-devel@nongnu.org; Tue, 01 Mar 2022 03:39:47 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]:45214) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nOy2r-0007s7-7b for qemu-devel@nongnu.org; Tue, 01 Mar 2022 03:39:46 -0500 Received: from mail-pf1-f198.google.com (mail-pf1-f198.google.com [209.85.210.198]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-203-7ozWoVi0N9KLb_oet4chWg-1; Tue, 01 Mar 2022 03:39:43 -0500 Received: by mail-pf1-f198.google.com with SMTP id x27-20020aa79a5b000000b004f3f7c2981eso4677990pfj.3 for ; Tue, 01 Mar 2022 00:39:43 -0800 (PST) Received: from localhost.localdomain ([94.177.118.144]) by smtp.gmail.com with ESMTPSA id l1-20020a17090aec0100b001bc6d8bb27dsm1439987pjy.37.2022.03.01.00.39.39 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Tue, 01 Mar 2022 00:39:41 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1646123984; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=OWAN/28atpuNSo2Qw825H0Y0k4mf5B5fhGEWjBM5WYE=; b=h3yL+bcQUF9h9qvvSiZvr9zNWYJ+GiS2KhO0XyIErCCdsqGr55hLnVKbXGKwQzsOsALYH3 XWoc1D2ecIfwvJVIG3NWYlUYWeO1Dphp2sKdlNeHdlDI1dGbpxrqxxQKRePdhj+pUOyRYk b8gJ6D1Kflcux3mUyJ/rIM7x+yLUR6s= X-MC-Unique: 7ozWoVi0N9KLb_oet4chWg-1 X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=OWAN/28atpuNSo2Qw825H0Y0k4mf5B5fhGEWjBM5WYE=; b=p0TuRfcInITfhXsavoVtabBE62pJqxmj3ew3zrF62u/1QkYFHDRlZizBXmcSJrit5l mrJLOs32RqSfk/d4yCsfm1XjSfq5gJKt6dkRnEbCDMc5wbDu9Bq7dxzRbJiI03Z8sPVL gWCnDjZcyhxHhllYZOcZ2RYDKAGfs5wzjbsOZcvbt5ABkIpRgYWJLw5Pmgnro16xSi7J Ua+vZyR3EwX1GlhwW2jHfbfxw1zpQ4Zm0/y1pgOnc3qjfMqAmFB8wSMoovvIks0Gc1jQ /T9zul0X+aiA81tWwqrUIq8E639MDb9xR3+p7bieFtnp0mAFFj0vehJZm8wcb3T0++yt aNsw== X-Gm-Message-State: AOAM5307VoB5+ddiDnDSYSsCVvB/zy0cawGRa7av1+qgyYggw5kuBoBr i9t4esROzV216A2LrJVDBhip047YDe9u9ND256KsVVhy+/pHjRubpRGUy3eYOuFFBhDKbwGiNS4 UqbF2RCNk6F900ZubwIyBwFrUUymKsy74zhLTvFFRV8PwPuacAjDF3bqZzToGevzf X-Received: by 2002:a63:d348:0:b0:342:3bc2:9b29 with SMTP id u8-20020a63d348000000b003423bc29b29mr20393179pgi.584.1646123982423; Tue, 01 Mar 2022 00:39:42 -0800 (PST) X-Google-Smtp-Source: ABdhPJyfJy4x3ZRzVa55RwrNDta9vJX/y9u21R8cKsqr4r8H1DAsw0v/rCh4KXl1eWIRdlRpGLUK0Q== X-Received: by 2002:a63:d348:0:b0:342:3bc2:9b29 with SMTP id u8-20020a63d348000000b003423bc29b29mr20393153pgi.584.1646123981865; Tue, 01 Mar 2022 00:39:41 -0800 (PST) From: Peter Xu To: qemu-devel@nongnu.org Subject: [PATCH v2 03/25] migration: Tracepoint change in postcopy-run bottom half Date: Tue, 1 Mar 2022 16:39:03 +0800 Message-Id: <20220301083925.33483-4-peterx@redhat.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20220301083925.33483-1-peterx@redhat.com> References: <20220301083925.33483-1-peterx@redhat.com> MIME-Version: 1.0 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=peterx@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=peterx@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -28 X-Spam_score: -2.9 X-Spam_bar: -- X-Spam_report: (-2.9 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.082, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H5=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Juan Quintela , "Dr . David Alan Gilbert" , peterx@redhat.com, Leonardo Bras Soares Passos Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZM-MESSAGEID: 1646124601374100001 Content-Type: text/plain; charset="utf-8" Remove the old two tracepoints and they're even near each other: trace_loadvm_postcopy_handle_run_cpu_sync() trace_loadvm_postcopy_handle_run_vmstart() Add trace_loadvm_postcopy_handle_run_bh() with a finer granule trace. Reviewed-by: Dr. David Alan Gilbert Signed-off-by: Peter Xu --- migration/savevm.c | 12 +++++++++--- migration/trace-events | 3 +-- 2 files changed, 10 insertions(+), 5 deletions(-) diff --git a/migration/savevm.c b/migration/savevm.c index 190cc5fc42..41e3238798 100644 --- a/migration/savevm.c +++ b/migration/savevm.c @@ -2006,13 +2006,19 @@ static void loadvm_postcopy_handle_run_bh(void *opa= que) Error *local_err =3D NULL; MigrationIncomingState *mis =3D opaque; =20 + trace_loadvm_postcopy_handle_run_bh("enter"); + /* TODO we should move all of this lot into postcopy_ram.c or a shared= code * in migration.c */ cpu_synchronize_all_post_init(); =20 + trace_loadvm_postcopy_handle_run_bh("after cpu sync"); + qemu_announce_self(&mis->announce_timer, migrate_announce_params()); =20 + trace_loadvm_postcopy_handle_run_bh("after announce"); + /* Make sure all file formats flush their mutable metadata. * If we get an error here, just don't restart the VM yet. */ bdrv_invalidate_cache_all(&local_err); @@ -2022,9 +2028,7 @@ static void loadvm_postcopy_handle_run_bh(void *opaqu= e) autostart =3D false; } =20 - trace_loadvm_postcopy_handle_run_cpu_sync(); - - trace_loadvm_postcopy_handle_run_vmstart(); + trace_loadvm_postcopy_handle_run_bh("after invalidate cache"); =20 dirty_bitmap_mig_before_vm_start(); =20 @@ -2037,6 +2041,8 @@ static void loadvm_postcopy_handle_run_bh(void *opaqu= e) } =20 qemu_bh_delete(mis->bh); + + trace_loadvm_postcopy_handle_run_bh("return"); } =20 /* After all discards we can start running and asking for pages */ diff --git a/migration/trace-events b/migration/trace-events index 92596c00d8..1aec580e92 100644 --- a/migration/trace-events +++ b/migration/trace-events @@ -16,8 +16,7 @@ loadvm_handle_recv_bitmap(char *s) "%s" loadvm_postcopy_handle_advise(void) "" loadvm_postcopy_handle_listen(const char *str) "%s" loadvm_postcopy_handle_run(void) "" -loadvm_postcopy_handle_run_cpu_sync(void) "" -loadvm_postcopy_handle_run_vmstart(void) "" +loadvm_postcopy_handle_run_bh(const char *str) "%s" loadvm_postcopy_handle_resume(void) "" loadvm_postcopy_ram_handle_discard(void) "" loadvm_postcopy_ram_handle_discard_end(void) "" --=20 2.32.0 From nobody Mon Feb 9 00:30:16 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1646124238201849.7021220928227; Tue, 1 Mar 2022 00:43:58 -0800 (PST) Received: from localhost ([::1]:41630 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nOy6u-0007qC-9N for importer@patchew.org; Tue, 01 Mar 2022 03:43:56 -0500 Received: from eggs.gnu.org ([209.51.188.92]:54170) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nOy2w-0003ak-Pe for qemu-devel@nongnu.org; Tue, 01 Mar 2022 03:39:50 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]:48106) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nOy2u-0007tD-GA for qemu-devel@nongnu.org; Tue, 01 Mar 2022 03:39:50 -0500 Received: from mail-pl1-f200.google.com (mail-pl1-f200.google.com [209.85.214.200]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-59-g8Jw9RFGNQOO2hyQvoGmFQ-1; Tue, 01 Mar 2022 03:39:47 -0500 Received: by mail-pl1-f200.google.com with SMTP id g2-20020a170902740200b0014fc971527eso5938267pll.14 for ; Tue, 01 Mar 2022 00:39:46 -0800 (PST) Received: from localhost.localdomain ([94.177.118.144]) by smtp.gmail.com with ESMTPSA id l1-20020a17090aec0100b001bc6d8bb27dsm1439987pjy.37.2022.03.01.00.39.42 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Tue, 01 Mar 2022 00:39:44 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1646123987; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=daQjwSg/qhTZylq18Eaj4UjQIvJ2zaoZ7Y1iN8dncXM=; b=Etk++X0ihUapD39FceZJnl6343o48SDghtKIoxvBf/zhAaxeZGAwjJbev/jRb4sGa7bWdT 0+qfC3lsfyMwo2pvHJQE2OB/vhM+g4IzJbSvP5+Tn9jJ20/vkRcpaMlyTVpkQA34yLNksd YOEuxadVdj5fe0GsO5Fh6rsJSUp+vcE= X-MC-Unique: g8Jw9RFGNQOO2hyQvoGmFQ-1 X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=daQjwSg/qhTZylq18Eaj4UjQIvJ2zaoZ7Y1iN8dncXM=; b=EDcu1yyvQgTo59m0L8knDdBFTlviTuN2jRbN6nzmZUK2Kx/FEZ14Fmadm7I+Hby9UM msMgzfP072dAfAkqU6GMijhX+fwx0sz9kfRd5OGRgsKQvz7me5jOn50z0FCEu8d28KdA Kp4gdKbVUzd3qtIWSWR/2KbSMoXIyj43k0d0VNlrCk/Aa3NHpCsIwlacVVwC2XX6ecmX 1v2Y6VmTu11ow1t1igTaU15uU9TAypHh8R3AxjRZOdMnfH0T9XlwcYIzPmIYDWa1TfcQ Qi2EbNUura992hj9gRiCZ5gz0UBe0IICovTlv/yx7OVGB0p6uZBLAUzJiDB8Sd1CIaJy an6Q== X-Gm-Message-State: AOAM532ERGBCLMJc02cmwg4zjokbmframM+DOhngV4IWc5gcmATsXFqJ ZFPFl0e2R5/IW2uIhNd/KraOtsTVNnhaBqTlsXdM8BGaW2XboWc/cFYeNRxRBsJOhrpCo3ayFeC ICDx6jBOCw6UmUMCH5Gdd1y09TBcOHcuUpUpEOzTyXR95QpObo0AI97GEmmLJjLEv X-Received: by 2002:a17:90a:d681:b0:1be:e3e5:3e6e with SMTP id x1-20020a17090ad68100b001bee3e53e6emr756591pju.122.1646123985616; Tue, 01 Mar 2022 00:39:45 -0800 (PST) X-Google-Smtp-Source: ABdhPJy/gs3QkU5Bgz79Qsk66/ZQo7WjpwkTAw+/l4MxHjofxRqflvZrV9sJjunlw2xAoRL3TiVF8g== X-Received: by 2002:a17:90a:d681:b0:1be:e3e5:3e6e with SMTP id x1-20020a17090ad68100b001bee3e53e6emr756551pju.122.1646123985012; Tue, 01 Mar 2022 00:39:45 -0800 (PST) From: Peter Xu To: qemu-devel@nongnu.org Subject: [PATCH v2 04/25] migration: Introduce postcopy channels on dest node Date: Tue, 1 Mar 2022 16:39:04 +0800 Message-Id: <20220301083925.33483-5-peterx@redhat.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20220301083925.33483-1-peterx@redhat.com> References: <20220301083925.33483-1-peterx@redhat.com> MIME-Version: 1.0 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=peterx@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=peterx@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -28 X-Spam_score: -2.9 X-Spam_bar: -- X-Spam_report: (-2.9 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.082, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H5=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Juan Quintela , "Dr . David Alan Gilbert" , peterx@redhat.com, Leonardo Bras Soares Passos Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZM-MESSAGEID: 1646124240277100001 Content-Type: text/plain; charset="utf-8" Postcopy handles huge pages in a special way that currently we can only have one "channel" to transfer the page. It's because when we install pages using UFFDIO_COPY, we need to have the w= hole huge page ready, it also means we need to have a temp huge page when trying= to receive the whole content of the page. Currently all maintainance around this tmp page is global: firstly we'll allocate a temp huge page, then we maintain its status mostly within ram_load_postcopy(). To enable multiple channels for postcopy, the first thing we need to do is = to prepare N temp huge pages as caching, one for each channel. Meanwhile we need to maintain the tmp huge page status per-channel too. To give some example, some local variables maintained in ram_load_postcopy() are listed; they are responsible for maintaining temp huge page status: - all_zero: this keeps whether this huge page contains all zeros - target_pages: this counts how many target pages have been copied - host_page: this keeps the host ptr for the page to install Move all these fields to be together with the temp huge pages to form a new structure called PostcopyTmpPage. Then for each (future) postcopy channel,= we need one structure to keep the state around. For vanilla postcopy, obviously there's only one channel. It contains both precopy and postcopy pages. This patch teaches the dest migration node to start realize the possible nu= mber of postcopy channels by introducing the "postcopy_channels" variable. Its value is calculated when setup postcopy on dest node (during POSTCOPY_LISTEN phase). Vanilla postcopy will have channels=3D1, but when postcopy-preempt capabili= ty is enabled (in the future), we will boost it to 2 because even during partial sending of a precopy huge page we still want to preempt it and start sending the postcopy requested page right away (so we start to keep two temp huge pages; more if we want to enable multifd). In this patch there's a TODO ma= rked for that; so far the channels is always set to 1. We need to send one "host huge page" on one channel only and we cannot split them, because otherwise the data upon the same huge page can locate on more than one channel so we need more complicated logic to manage. One temp host huge page for each channel will be enough for us for now. Postcopy will still always use the index=3D0 huge page even after this patc= h. However it prepares for the latter patches where it can start to use multip= le channels (which needs src intervention, because only src knows which channe= l we should use). Reviewed-by: Dr. David Alan Gilbert Signed-off-by: Peter Xu --- migration/migration.h | 36 +++++++++++++++++++++++- migration/postcopy-ram.c | 60 ++++++++++++++++++++++++++++++---------- migration/ram.c | 43 ++++++++++++++-------------- migration/savevm.c | 12 ++++++++ 4 files changed, 113 insertions(+), 38 deletions(-) diff --git a/migration/migration.h b/migration/migration.h index 8130b703eb..42c7395094 100644 --- a/migration/migration.h +++ b/migration/migration.h @@ -45,6 +45,24 @@ struct PostcopyBlocktimeContext; */ #define CLEAR_BITMAP_SHIFT_MAX 31 =20 +/* This is an abstraction of a "temp huge page" for postcopy's purpose */ +typedef struct { + /* + * This points to a temporary huge page as a buffer for UFFDIO_COPY. = It's + * mmap()ed and needs to be freed when cleanup. + */ + void *tmp_huge_page; + /* + * This points to the host page we're going to install for this temp p= age. + * It tells us after we've received the whole page, where we should pu= t it. + */ + void *host_addr; + /* Number of small pages copied (in size of TARGET_PAGE_SIZE) */ + unsigned int target_pages; + /* Whether this page contains all zeros */ + bool all_zero; +} PostcopyTmpPage; + /* State for the incoming migration */ struct MigrationIncomingState { QEMUFile *from_src_file; @@ -81,7 +99,22 @@ struct MigrationIncomingState { QemuMutex rp_mutex; /* We send replies from multiple threads */ /* RAMBlock of last request sent to source */ RAMBlock *last_rb; - void *postcopy_tmp_page; + /* + * Number of postcopy channels including the default precopy channel, = so + * vanilla postcopy will only contain one channel which contain both + * precopy and postcopy streams. + * + * This is calculated when the src requests to enable postcopy but bef= ore + * it starts. Its value can depend on e.g. whether postcopy preemptio= n is + * enabled. + */ + unsigned int postcopy_channels; + /* + * An array of temp host huge pages to be used, one for each postcopy + * channel. + */ + PostcopyTmpPage *postcopy_tmp_pages; + /* This is shared for all postcopy channels */ void *postcopy_tmp_zero_page; /* PostCopyFD's for external userfaultfds & handlers of shared memory = */ GArray *postcopy_remote_fds; @@ -391,5 +424,6 @@ bool migration_rate_limit(void); void migration_cancel(const Error *error); =20 void populate_vfio_info(MigrationInfo *info); +void postcopy_temp_page_reset(PostcopyTmpPage *tmp_page); =20 #endif diff --git a/migration/postcopy-ram.c b/migration/postcopy-ram.c index 2a2cc5faf8..30c3508f44 100644 --- a/migration/postcopy-ram.c +++ b/migration/postcopy-ram.c @@ -526,9 +526,18 @@ int postcopy_ram_incoming_init(MigrationIncomingState = *mis) =20 static void postcopy_temp_pages_cleanup(MigrationIncomingState *mis) { - if (mis->postcopy_tmp_page) { - munmap(mis->postcopy_tmp_page, mis->largest_page_size); - mis->postcopy_tmp_page =3D NULL; + int i; + + if (mis->postcopy_tmp_pages) { + for (i =3D 0; i < mis->postcopy_channels; i++) { + if (mis->postcopy_tmp_pages[i].tmp_huge_page) { + munmap(mis->postcopy_tmp_pages[i].tmp_huge_page, + mis->largest_page_size); + mis->postcopy_tmp_pages[i].tmp_huge_page =3D NULL; + } + } + g_free(mis->postcopy_tmp_pages); + mis->postcopy_tmp_pages =3D NULL; } =20 if (mis->postcopy_tmp_zero_page) { @@ -1092,17 +1101,30 @@ retry: =20 static int postcopy_temp_pages_setup(MigrationIncomingState *mis) { - int err; - - mis->postcopy_tmp_page =3D mmap(NULL, mis->largest_page_size, - PROT_READ | PROT_WRITE, - MAP_PRIVATE | MAP_ANONYMOUS, -1, 0); - if (mis->postcopy_tmp_page =3D=3D MAP_FAILED) { - err =3D errno; - mis->postcopy_tmp_page =3D NULL; - error_report("%s: Failed to map postcopy_tmp_page %s", - __func__, strerror(err)); - return -err; + PostcopyTmpPage *tmp_page; + int err, i, channels; + void *temp_page; + + /* TODO: will be boosted when enable postcopy preemption */ + mis->postcopy_channels =3D 1; + + channels =3D mis->postcopy_channels; + mis->postcopy_tmp_pages =3D g_malloc0_n(sizeof(PostcopyTmpPage), chann= els); + + for (i =3D 0; i < channels; i++) { + tmp_page =3D &mis->postcopy_tmp_pages[i]; + temp_page =3D mmap(NULL, mis->largest_page_size, PROT_READ | PROT_= WRITE, + MAP_PRIVATE | MAP_ANONYMOUS, -1, 0); + if (temp_page =3D=3D MAP_FAILED) { + err =3D errno; + error_report("%s: Failed to map postcopy_tmp_pages[%d]: %s", + __func__, i, strerror(err)); + /* Clean up will be done later */ + return -err; + } + tmp_page->tmp_huge_page =3D temp_page; + /* Initialize default states for each tmp page */ + postcopy_temp_page_reset(tmp_page); } =20 /* @@ -1352,6 +1374,16 @@ int postcopy_wake_shared(struct PostCopyFD *pcfd, #endif =20 /* -----------------------------------------------------------------------= -- */ +void postcopy_temp_page_reset(PostcopyTmpPage *tmp_page) +{ + tmp_page->target_pages =3D 0; + tmp_page->host_addr =3D NULL; + /* + * This is set to true when reset, and cleared as long as we received = any + * of the non-zero small page within this huge page. + */ + tmp_page->all_zero =3D true; +} =20 void postcopy_fault_thread_notify(MigrationIncomingState *mis) { diff --git a/migration/ram.c b/migration/ram.c index 781f0745dc..0fc6b8e349 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -3641,11 +3641,8 @@ static int ram_load_postcopy(QEMUFile *f) bool place_needed =3D false; bool matches_target_page_size =3D false; MigrationIncomingState *mis =3D migration_incoming_get_current(); - /* Temporary page that is later 'placed' */ - void *postcopy_host_page =3D mis->postcopy_tmp_page; - void *host_page =3D NULL; - bool all_zero =3D true; - int target_pages =3D 0; + /* Currently we only use channel 0. TODO: use all the channels */ + PostcopyTmpPage *tmp_page =3D &mis->postcopy_tmp_pages[0]; =20 while (!ret && !(flags & RAM_SAVE_FLAG_EOS)) { ram_addr_t addr; @@ -3689,7 +3686,7 @@ static int ram_load_postcopy(QEMUFile *f) ret =3D -EINVAL; break; } - target_pages++; + tmp_page->target_pages++; matches_target_page_size =3D block->page_size =3D=3D TARGET_PA= GE_SIZE; /* * Postcopy requires that we place whole host pages atomically; @@ -3701,15 +3698,16 @@ static int ram_load_postcopy(QEMUFile *f) * however the source ensures it always sends all the componen= ts * of a host page in one chunk. */ - page_buffer =3D postcopy_host_page + + page_buffer =3D tmp_page->tmp_huge_page + host_page_offset_from_ram_block_offset(block, ad= dr); /* If all TP are zero then we can optimise the place */ - if (target_pages =3D=3D 1) { - host_page =3D host_page_from_ram_block_offset(block, addr); - } else if (host_page !=3D host_page_from_ram_block_offset(bloc= k, - addr))= { + if (tmp_page->target_pages =3D=3D 1) { + tmp_page->host_addr =3D + host_page_from_ram_block_offset(block, addr); + } else if (tmp_page->host_addr !=3D + host_page_from_ram_block_offset(block, addr)) { /* not the 1st TP within the HP */ - error_report("Non-same host page %p/%p", host_page, + error_report("Non-same host page %p/%p", tmp_page->host_ad= dr, host_page_from_ram_block_offset(block, addr)); ret =3D -EINVAL; break; @@ -3719,10 +3717,11 @@ static int ram_load_postcopy(QEMUFile *f) * If it's the last part of a host page then we place the host * page */ - if (target_pages =3D=3D (block->page_size / TARGET_PAGE_SIZE))= { + if (tmp_page->target_pages =3D=3D + (block->page_size / TARGET_PAGE_SIZE)) { place_needed =3D true; } - place_source =3D postcopy_host_page; + place_source =3D tmp_page->tmp_huge_page; } =20 switch (flags & ~RAM_SAVE_FLAG_CONTINUE) { @@ -3736,12 +3735,12 @@ static int ram_load_postcopy(QEMUFile *f) memset(page_buffer, ch, TARGET_PAGE_SIZE); } if (ch) { - all_zero =3D false; + tmp_page->all_zero =3D false; } break; =20 case RAM_SAVE_FLAG_PAGE: - all_zero =3D false; + tmp_page->all_zero =3D false; if (!matches_target_page_size) { /* For huge pages, we always use temporary buffer */ qemu_get_buffer(f, page_buffer, TARGET_PAGE_SIZE); @@ -3759,7 +3758,7 @@ static int ram_load_postcopy(QEMUFile *f) } break; case RAM_SAVE_FLAG_COMPRESS_PAGE: - all_zero =3D false; + tmp_page->all_zero =3D false; len =3D qemu_get_be32(f); if (len < 0 || len > compressBound(TARGET_PAGE_SIZE)) { error_report("Invalid compressed data length: %d", len); @@ -3791,16 +3790,14 @@ static int ram_load_postcopy(QEMUFile *f) } =20 if (!ret && place_needed) { - if (all_zero) { - ret =3D postcopy_place_page_zero(mis, host_page, block); + if (tmp_page->all_zero) { + ret =3D postcopy_place_page_zero(mis, tmp_page->host_addr,= block); } else { - ret =3D postcopy_place_page(mis, host_page, place_source, + ret =3D postcopy_place_page(mis, tmp_page->host_addr, plac= e_source, block); } place_needed =3D false; - target_pages =3D 0; - /* Assume we have a zero page until we detect something differ= ent */ - all_zero =3D true; + postcopy_temp_page_reset(tmp_page); } } =20 diff --git a/migration/savevm.c b/migration/savevm.c index 41e3238798..0ccd7e5e3f 100644 --- a/migration/savevm.c +++ b/migration/savevm.c @@ -2579,6 +2579,18 @@ void qemu_loadvm_state_cleanup(void) /* Return true if we should continue the migration, or false. */ static bool postcopy_pause_incoming(MigrationIncomingState *mis) { + int i; + + /* + * If network is interrupted, any temp page we received will be useless + * because we didn't mark them as "received" in receivedmap. After a + * proper recovery later (which will sync src dirty bitmap with receiv= edmap + * on dest) these cached small pages will be resent again. + */ + for (i =3D 0; i < mis->postcopy_channels; i++) { + postcopy_temp_page_reset(&mis->postcopy_tmp_pages[i]); + } + trace_postcopy_pause_incoming(); =20 assert(migrate_postcopy_ram()); --=20 2.32.0 From nobody Mon Feb 9 00:30:16 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1646124619835155.68082678096914; Tue, 1 Mar 2022 00:50:19 -0800 (PST) Received: from localhost ([::1]:48170 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nOyD4-00048e-Ov for importer@patchew.org; Tue, 01 Mar 2022 03:50:18 -0500 Received: from eggs.gnu.org ([209.51.188.92]:54210) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nOy2y-0003fM-D7 for qemu-devel@nongnu.org; Tue, 01 Mar 2022 03:39:52 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:45148) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nOy2w-0007uU-Ua for qemu-devel@nongnu.org; Tue, 01 Mar 2022 03:39:52 -0500 Received: from mail-pf1-f197.google.com (mail-pf1-f197.google.com [209.85.210.197]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-60-Eq6wgeT9MtmSnpr-siMsUw-1; Tue, 01 Mar 2022 03:39:49 -0500 Received: by mail-pf1-f197.google.com with SMTP id t134-20020a62788c000000b004e1367caccaso9357265pfc.14 for ; Tue, 01 Mar 2022 00:39:49 -0800 (PST) Received: from localhost.localdomain ([94.177.118.144]) by smtp.gmail.com with ESMTPSA id l1-20020a17090aec0100b001bc6d8bb27dsm1439987pjy.37.2022.03.01.00.39.45 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Tue, 01 Mar 2022 00:39:47 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1646123990; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=WVAvZ0HPGkUtqiJCQBEwgvy+s0e8LwsA8UQ0Uhpgy1E=; b=UsZi9F+VdpjdKwL3Tgg6uyQyC6kXLPGutL8nWiAzre9oHhBX9diu+MUrPi+YU/1GDovqdH TlSt6BX3/YlfF3kuVxX5Rx+Ac4Kjvzso1xPu4TZnqqbg3PTJcDoXAlWdjThsHDxZyEXgM9 IcAewFED5STffrja55Bp8PkuDIeId8M= X-MC-Unique: Eq6wgeT9MtmSnpr-siMsUw-1 X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=WVAvZ0HPGkUtqiJCQBEwgvy+s0e8LwsA8UQ0Uhpgy1E=; b=AGDgiASlQxQYXx3lojUdR498sm2nZePme3VojcVsninNJfBCCSMUUW1Ks3RfKw42Nd s7GCqxu4KM02J+lP0T0cttEPLatOxTsfgfh023Y89iir1eHoqki/RaCa/DSWgD/u6uxv InQZgQ0lH6gnSbDRboXv6euNHi0EzcOdikpDTc+WCwXCkU9PDQB27zt/ih2GgYOojmG5 tjKTiX5qfMLqnoCK9MNLM3X0z2R2+oZlFT0AQUlwNLPQNdTiJ1j+Umfi+tMoCZobS/4y GYl/XjyTLQK2fhp3sXXUipWErI776T8PmGst18aOOCCuaCz/SdydZFTs2hgKl6oPy1kG LxKw== X-Gm-Message-State: AOAM532tMfwbYTnZA028/V4R+Dr545O926usk/i5pPINkPARWS0/jx9T vCQgOwNoxAHGZnrtPXWeeBS478fBk/QplrHDSFVgLOGaOVL5p7a5bUwhX8tXULfrLnmwpWwagke CpDCJAg63LFE9D8SYpw1yHrMqrm72ntVjctCEW7vyAZbWlz0/NdE7ddRK9f+9A2tN X-Received: by 2002:a17:90a:10c8:b0:1bc:e369:1f2c with SMTP id b8-20020a17090a10c800b001bce3691f2cmr20578054pje.193.1646123988241; Tue, 01 Mar 2022 00:39:48 -0800 (PST) X-Google-Smtp-Source: ABdhPJx7eqSKl/oQ6mo67/dudsiE71eArMnnsBCW2n7s9UWCoR8g1COgRU1nVEswURLgYHaV9x12xQ== X-Received: by 2002:a17:90a:10c8:b0:1bc:e369:1f2c with SMTP id b8-20020a17090a10c800b001bce3691f2cmr20578038pje.193.1646123987883; Tue, 01 Mar 2022 00:39:47 -0800 (PST) From: Peter Xu To: qemu-devel@nongnu.org Subject: [PATCH v2 05/25] migration: Dump ramblock and offset too when non-same-page detected Date: Tue, 1 Mar 2022 16:39:05 +0800 Message-Id: <20220301083925.33483-6-peterx@redhat.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20220301083925.33483-1-peterx@redhat.com> References: <20220301083925.33483-1-peterx@redhat.com> MIME-Version: 1.0 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=peterx@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=peterx@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -28 X-Spam_score: -2.9 X-Spam_bar: -- X-Spam_report: (-2.9 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.082, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H5=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Juan Quintela , "Dr . David Alan Gilbert" , peterx@redhat.com, Leonardo Bras Soares Passos Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZM-MESSAGEID: 1646124621003100001 Content-Type: text/plain; charset="utf-8" In ram_load_postcopy() we'll try to detect non-same-page case and dump erro= r. This error is very helpful for debugging. Adding ramblock & offset into the error log too. Reviewed-by: Dr. David Alan Gilbert Signed-off-by: Peter Xu --- migration/ram.c | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-) diff --git a/migration/ram.c b/migration/ram.c index 0fc6b8e349..3a216c2340 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -3707,8 +3707,12 @@ static int ram_load_postcopy(QEMUFile *f) } else if (tmp_page->host_addr !=3D host_page_from_ram_block_offset(block, addr)) { /* not the 1st TP within the HP */ - error_report("Non-same host page %p/%p", tmp_page->host_ad= dr, - host_page_from_ram_block_offset(block, addr)); + error_report("Non-same host page detected. Target host pa= ge %p, " + "received host page %p " + "(rb %s offset 0x"RAM_ADDR_FMT" target_pages = %d)", + tmp_page->host_addr, + host_page_from_ram_block_offset(block, addr), + block->idstr, addr, tmp_page->target_pages); ret =3D -EINVAL; break; } --=20 2.32.0 From nobody Mon Feb 9 00:30:16 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1646124612232975.7935382276786; Tue, 1 Mar 2022 00:50:12 -0800 (PST) Received: from localhost ([::1]:48104 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nOyCx-000469-3h for importer@patchew.org; Tue, 01 Mar 2022 03:50:11 -0500 Received: from eggs.gnu.org ([209.51.188.92]:54276) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nOy31-0003py-O0 for qemu-devel@nongnu.org; Tue, 01 Mar 2022 03:39:55 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]:60781) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nOy2z-0007vN-W8 for qemu-devel@nongnu.org; Tue, 01 Mar 2022 03:39:55 -0500 Received: from mail-pg1-f199.google.com (mail-pg1-f199.google.com [209.85.215.199]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-571-0t_lI46vO2CzZR-SBY86ng-1; Tue, 01 Mar 2022 03:39:52 -0500 Received: by mail-pg1-f199.google.com with SMTP id bh9-20020a056a02020900b0036c0d29eb3eso8182014pgb.9 for ; Tue, 01 Mar 2022 00:39:52 -0800 (PST) Received: from localhost.localdomain ([94.177.118.144]) by smtp.gmail.com with ESMTPSA id l1-20020a17090aec0100b001bc6d8bb27dsm1439987pjy.37.2022.03.01.00.39.48 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Tue, 01 Mar 2022 00:39:50 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1646123993; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=w9F784ao0L88QQQR4mfL5Wnon20EdN8YOIdXyuzL9XE=; b=Tub/SKtS8hh2/2s9c6YxIKEcihLdQ+sUxJLPaxL+3iBny3Pc3YQLSQCqom3xVD8U+Xk0lE 5DHyI+sPbltcMLpfCqTthAi86PEJ5ljjBisnppzeQQw6SzcEXj9flHYA8jQs9IIsXkIPr0 cjyT0wv/zo4nxTapmQIkQSbKHn9dAG0= X-MC-Unique: 0t_lI46vO2CzZR-SBY86ng-1 X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=w9F784ao0L88QQQR4mfL5Wnon20EdN8YOIdXyuzL9XE=; b=690Prr2Qnu12CfEqmEvhSUS2Ll3P7qxGOhBTYb8A3c1MeZN1PDTG8AUFBWTpKMorFK F6f7ZijqBEJvFdSH4v/5wmGBbBHgkKXq7SohJf5hZtelJb1y5KaTXoPgUWmDB17oG8En yLyedJ6ALRSFZxTnRlF1950RLtpI3rQO6f/rA5VNHplym5EUulZ2u2JTSBuTeGQPX/XD XAyl1HPxCdORT1XMIwm/CDzhDoCbMJrDVPh8Sb29kvqUGNpTpFV7fysS/qz1xAdqhbOK yiJFZc9yIc7aUWuCwgJGTSSmb6SUhn/Cycklbq/s9XcWDUi5i+UTm0SPd9quLKNLEmd+ YvDw== X-Gm-Message-State: AOAM530LfGKaM6X7qveK3VqTvu0wk1/LdAXYMHouq0wOr9rBq+yr4T9G HP0Fz0nucUihzWPcDggzrz67aRLP81ZWtfswxO4RIq5uubcafCtPN4L0ppebfEBV08Z4vpY6ftn Xf37QvN5F1IHUlF/+cVuZ/foazAoorKvRDcdleCC8CZgZtzwWRSsGAXP4hhddZT34 X-Received: by 2002:a05:6a00:b92:b0:4f1:4b2:737f with SMTP id g18-20020a056a000b9200b004f104b2737fmr26259938pfj.31.1646123991285; Tue, 01 Mar 2022 00:39:51 -0800 (PST) X-Google-Smtp-Source: ABdhPJykQZwtUeAkLAf628SqiYQIL4pyL/IKYKn0IjQOnd0cA7Y95U9NeILoiGXIwHNriew3eushWA== X-Received: by 2002:a05:6a00:b92:b0:4f1:4b2:737f with SMTP id g18-20020a056a000b9200b004f104b2737fmr26259906pfj.31.1646123990852; Tue, 01 Mar 2022 00:39:50 -0800 (PST) From: Peter Xu To: qemu-devel@nongnu.org Subject: [PATCH v2 06/25] migration: Add postcopy_thread_create() Date: Tue, 1 Mar 2022 16:39:06 +0800 Message-Id: <20220301083925.33483-7-peterx@redhat.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20220301083925.33483-1-peterx@redhat.com> References: <20220301083925.33483-1-peterx@redhat.com> MIME-Version: 1.0 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=peterx@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=peterx@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -28 X-Spam_score: -2.9 X-Spam_bar: -- X-Spam_report: (-2.9 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.082, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H5=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Juan Quintela , "Dr . David Alan Gilbert" , peterx@redhat.com, Leonardo Bras Soares Passos Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZM-MESSAGEID: 1646124613097100003 Content-Type: text/plain; charset="utf-8" Postcopy create threads. A common manner is we init a sem and use it to sync with the thread. Namely, we have fault_thread_sem and listen_thread_sem and they're only used for this. Make it a shared infrastructure so it's easier to create yet another thread. Reviewed-by: Dr. David Alan Gilbert Signed-off-by: Peter Xu --- migration/migration.h | 8 +++++--- migration/postcopy-ram.c | 23 +++++++++++++++++------ migration/postcopy-ram.h | 4 ++++ migration/savevm.c | 12 +++--------- 4 files changed, 29 insertions(+), 18 deletions(-) diff --git a/migration/migration.h b/migration/migration.h index 42c7395094..8445e1d14a 100644 --- a/migration/migration.h +++ b/migration/migration.h @@ -70,7 +70,11 @@ struct MigrationIncomingState { /* A hook to allow cleanup at the end of incoming migration */ void *transport_data; void (*transport_cleanup)(void *data); - + /* + * Used to sync thread creations. Note that we can't create threads in + * parallel with this sem. + */ + QemuSemaphore thread_sync_sem; /* * Free at the start of the main state load, set as the main thread fi= nishes * loading state. @@ -83,13 +87,11 @@ struct MigrationIncomingState { size_t largest_page_size; bool have_fault_thread; QemuThread fault_thread; - QemuSemaphore fault_thread_sem; /* Set this when we want the fault thread to quit */ bool fault_thread_quit; =20 bool have_listen_thread; QemuThread listen_thread; - QemuSemaphore listen_thread_sem; =20 /* For the kernel to send us notifications */ int userfault_fd; diff --git a/migration/postcopy-ram.c b/migration/postcopy-ram.c index 30c3508f44..d08d396c63 100644 --- a/migration/postcopy-ram.c +++ b/migration/postcopy-ram.c @@ -78,6 +78,20 @@ int postcopy_notify(enum PostcopyNotifyReason reason, Er= ror **errp) &pnd); } =20 +/* + * NOTE: this routine is not thread safe, we can't call it concurrently. B= ut it + * should be good enough for migration's purposes. + */ +void postcopy_thread_create(MigrationIncomingState *mis, + QemuThread *thread, const char *name, + void *(*fn)(void *), int joinable) +{ + qemu_sem_init(&mis->thread_sync_sem, 0); + qemu_thread_create(thread, name, fn, mis, joinable); + qemu_sem_wait(&mis->thread_sync_sem); + qemu_sem_destroy(&mis->thread_sync_sem); +} + /* Postcopy needs to detect accesses to pages that haven't yet been copied * across, and efficiently map new pages in, the techniques for doing this * are target OS specific. @@ -902,7 +916,7 @@ static void *postcopy_ram_fault_thread(void *opaque) trace_postcopy_ram_fault_thread_entry(); rcu_register_thread(); mis->last_rb =3D NULL; /* last RAMBlock we sent part of */ - qemu_sem_post(&mis->fault_thread_sem); + qemu_sem_post(&mis->thread_sync_sem); =20 struct pollfd *pfd; size_t pfd_len =3D 2 + mis->postcopy_remote_fds->len; @@ -1173,11 +1187,8 @@ int postcopy_ram_incoming_setup(MigrationIncomingSta= te *mis) return -1; } =20 - qemu_sem_init(&mis->fault_thread_sem, 0); - qemu_thread_create(&mis->fault_thread, "postcopy/fault", - postcopy_ram_fault_thread, mis, QEMU_THREAD_JOINABL= E); - qemu_sem_wait(&mis->fault_thread_sem); - qemu_sem_destroy(&mis->fault_thread_sem); + postcopy_thread_create(mis, &mis->fault_thread, "postcopy/fault", + postcopy_ram_fault_thread, QEMU_THREAD_JOINABLE= ); mis->have_fault_thread =3D true; =20 /* Mark so that we get notified of accesses to unwritten areas */ diff --git a/migration/postcopy-ram.h b/migration/postcopy-ram.h index 6d2b3cf124..07684c0e1d 100644 --- a/migration/postcopy-ram.h +++ b/migration/postcopy-ram.h @@ -135,6 +135,10 @@ void postcopy_remove_notifier(NotifierWithReturn *n); /* Call the notifier list set by postcopy_add_start_notifier */ int postcopy_notify(enum PostcopyNotifyReason reason, Error **errp); =20 +void postcopy_thread_create(MigrationIncomingState *mis, + QemuThread *thread, const char *name, + void *(*fn)(void *), int joinable); + struct PostCopyFD; =20 /* ufd is a pointer to the struct uffd_msg *TODO: more Portable! */ diff --git a/migration/savevm.c b/migration/savevm.c index 0ccd7e5e3f..967ff80547 100644 --- a/migration/savevm.c +++ b/migration/savevm.c @@ -1863,7 +1863,7 @@ static void *postcopy_ram_listen_thread(void *opaque) =20 migrate_set_state(&mis->state, MIGRATION_STATUS_ACTIVE, MIGRATION_STATUS_POSTCOPY_ACTIVE); - qemu_sem_post(&mis->listen_thread_sem); + qemu_sem_post(&mis->thread_sync_sem); trace_postcopy_ram_listen_thread_start(); =20 rcu_register_thread(); @@ -1988,14 +1988,8 @@ static int loadvm_postcopy_handle_listen(MigrationIn= comingState *mis) } =20 mis->have_listen_thread =3D true; - /* Start up the listening thread and wait for it to signal ready */ - qemu_sem_init(&mis->listen_thread_sem, 0); - qemu_thread_create(&mis->listen_thread, "postcopy/listen", - postcopy_ram_listen_thread, NULL, - QEMU_THREAD_DETACHED); - qemu_sem_wait(&mis->listen_thread_sem); - qemu_sem_destroy(&mis->listen_thread_sem); - + postcopy_thread_create(mis, &mis->listen_thread, "postcopy/listen", + postcopy_ram_listen_thread, QEMU_THREAD_DETACHE= D); trace_loadvm_postcopy_handle_listen("return"); =20 return 0; --=20 2.32.0 From nobody Mon Feb 9 00:30:16 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 164612490182016.052234099553857; Tue, 1 Mar 2022 00:55:01 -0800 (PST) Received: from localhost ([::1]:56828 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nOyHc-0001jd-DE for importer@patchew.org; Tue, 01 Mar 2022 03:55:00 -0500 Received: from eggs.gnu.org ([209.51.188.92]:54328) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nOy34-0003yF-Fc for qemu-devel@nongnu.org; Tue, 01 Mar 2022 03:39:58 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:33875) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nOy32-0007wE-Ok for qemu-devel@nongnu.org; Tue, 01 Mar 2022 03:39:58 -0500 Received: from mail-pf1-f197.google.com (mail-pf1-f197.google.com [209.85.210.197]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-384-NeL7EI1hMNiA6l6GwRmFZw-1; Tue, 01 Mar 2022 03:39:55 -0500 Received: by mail-pf1-f197.google.com with SMTP id d132-20020a621d8a000000b004f41f34db96so1930322pfd.2 for ; Tue, 01 Mar 2022 00:39:55 -0800 (PST) Received: from localhost.localdomain ([94.177.118.144]) by smtp.gmail.com with ESMTPSA id l1-20020a17090aec0100b001bc6d8bb27dsm1439987pjy.37.2022.03.01.00.39.51 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Tue, 01 Mar 2022 00:39:53 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1646123996; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=PhXk+S58P3eb4DnKsbjdNHFaBnOZp2LgpOZHkR2wIBo=; b=FhBqOMPrNNHzj19hJPvezOK1vEfSVCyfmLBvgPNEObivPUup2pLyUButci4BZQ6KgbB7dI 7cIbyNJ0VbSOBlA5DpNSfwLT8wUGXXSm7q0q9mlp752ladSACsKCsZTObz9+lJpS+g6Bx+ mtMu+4/cHO1m7Cc9s0IXV8qQIFMhM6M= X-MC-Unique: NeL7EI1hMNiA6l6GwRmFZw-1 X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=PhXk+S58P3eb4DnKsbjdNHFaBnOZp2LgpOZHkR2wIBo=; b=PAhXrSL6ALGY0OiQwOwR2u4WzKL8K4fusRVN2geakwa/AQT5iB3DLtoNtvcLUZ+f/p EqJwJlAjpgBFXK1loQY+xmQtA3MjjHfHn5VF0hyxPzD9YvxCMip29771YkGGNlz5QhwD ChMnnOqfpGZUvOHzgGyyPYrwW1VsxSsdllrbT9sIDIQx/awSnhhstBCduvQAWIWa4CP4 +PzodyUlX+qQu0lSaHkNxEEok/25nV8oSPrR1+GGmoPm4JSCerizLk8RqMMGII1Gtbtr maLSiR5nWlQmMCX0RIL9HikLTMhn7+KyIyOGoOzdfN882asNe5nWsrMzLsPCVTmh1q2W n7Og== X-Gm-Message-State: AOAM53099vdVxMp1oLgNb9eqJjFQSZAwBg2i3S95+Cc9UKPxawBk8LCi qzIYtclXcRMef7exqShcAoyX8TJkbax4nkj5kBM/vroQg3HjmiqU+b6CoZidptOhGZfIaWGI69w DD0D0k19jTWzqFdL+gMqQAcC4o2CuO8KSTzFsmKtl0/NkMkJrwqYh5WLIrPbhA6Kk X-Received: by 2002:a05:6a02:193:b0:375:65a5:2fcd with SMTP id bj19-20020a056a02019300b0037565a52fcdmr20706691pgb.288.1646123994068; Tue, 01 Mar 2022 00:39:54 -0800 (PST) X-Google-Smtp-Source: ABdhPJzxh7+8QOo8K0IW9pJbtjcnZJHfhEmUCQcY1DEwwQPvMX7J5tHVt0vFLGqCaBOHsfgTM0eGxg== X-Received: by 2002:a05:6a02:193:b0:375:65a5:2fcd with SMTP id bj19-20020a056a02019300b0037565a52fcdmr20706663pgb.288.1646123993652; Tue, 01 Mar 2022 00:39:53 -0800 (PST) From: Peter Xu To: qemu-devel@nongnu.org Subject: [PATCH v2 07/25] migration: Move static var in ram_block_from_stream() into global Date: Tue, 1 Mar 2022 16:39:07 +0800 Message-Id: <20220301083925.33483-8-peterx@redhat.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20220301083925.33483-1-peterx@redhat.com> References: <20220301083925.33483-1-peterx@redhat.com> MIME-Version: 1.0 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=peterx@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=peterx@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -28 X-Spam_score: -2.9 X-Spam_bar: -- X-Spam_report: (-2.9 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.082, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H5=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Juan Quintela , "Dr . David Alan Gilbert" , peterx@redhat.com, Leonardo Bras Soares Passos Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZM-MESSAGEID: 1646124903147100003 Content-Type: text/plain; charset="utf-8" Static variable is very unfriendly to threading of ram_block_from_stream(). Move it into MigrationIncomingState. Make the incoming state pointer to be passed over to ram_block_from_stream(= ) on both caller sites. Reviewed-by: Dr. David Alan Gilbert Signed-off-by: Peter Xu --- migration/migration.h | 3 ++- migration/ram.c | 13 +++++++++---- 2 files changed, 11 insertions(+), 5 deletions(-) diff --git a/migration/migration.h b/migration/migration.h index 8445e1d14a..d8b9850eae 100644 --- a/migration/migration.h +++ b/migration/migration.h @@ -66,7 +66,8 @@ typedef struct { /* State for the incoming migration */ struct MigrationIncomingState { QEMUFile *from_src_file; - + /* Previously received RAM's RAMBlock pointer */ + RAMBlock *last_recv_block; /* A hook to allow cleanup at the end of incoming migration */ void *transport_data; void (*transport_cleanup)(void *data); diff --git a/migration/ram.c b/migration/ram.c index 3a216c2340..9516dd655a 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -3185,12 +3185,14 @@ static int load_xbzrle(QEMUFile *f, ram_addr_t addr= , void *host) * * Returns a pointer from within the RCU-protected ram_list. * + * @mis: the migration incoming state pointer * @f: QEMUFile where to read the data from * @flags: Page flags (mostly to see if it's a continuation of previous bl= ock) */ -static inline RAMBlock *ram_block_from_stream(QEMUFile *f, int flags) +static inline RAMBlock *ram_block_from_stream(MigrationIncomingState *mis, + QEMUFile *f, int flags) { - static RAMBlock *block; + RAMBlock *block =3D mis->last_recv_block; char id[256]; uint8_t len; =20 @@ -3217,6 +3219,8 @@ static inline RAMBlock *ram_block_from_stream(QEMUFil= e *f, int flags) return NULL; } =20 + mis->last_recv_block =3D block; + return block; } =20 @@ -3669,7 +3673,7 @@ static int ram_load_postcopy(QEMUFile *f) trace_ram_load_postcopy_loop((uint64_t)addr, flags); if (flags & (RAM_SAVE_FLAG_ZERO | RAM_SAVE_FLAG_PAGE | RAM_SAVE_FLAG_COMPRESS_PAGE)) { - block =3D ram_block_from_stream(f, flags); + block =3D ram_block_from_stream(mis, f, flags); if (!block) { ret =3D -EINVAL; break; @@ -3881,6 +3885,7 @@ void colo_flush_ram_cache(void) */ static int ram_load_precopy(QEMUFile *f) { + MigrationIncomingState *mis =3D migration_incoming_get_current(); int flags =3D 0, ret =3D 0, invalid_flags =3D 0, len =3D 0, i =3D 0; /* ADVISE is earlier, it shows the source has the postcopy capability = on */ bool postcopy_advised =3D postcopy_is_advised(); @@ -3919,7 +3924,7 @@ static int ram_load_precopy(QEMUFile *f) =20 if (flags & (RAM_SAVE_FLAG_ZERO | RAM_SAVE_FLAG_PAGE | RAM_SAVE_FLAG_COMPRESS_PAGE | RAM_SAVE_FLAG_XBZRLE)) { - RAMBlock *block =3D ram_block_from_stream(f, flags); + RAMBlock *block =3D ram_block_from_stream(mis, f, flags); =20 host =3D host_from_ram_block_offset(block, addr); /* --=20 2.32.0 From nobody Mon Feb 9 00:30:16 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1646124891584836.0106223901071; Tue, 1 Mar 2022 00:54:51 -0800 (PST) Received: from localhost ([::1]:56500 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nOyHS-0001Vf-DE for importer@patchew.org; Tue, 01 Mar 2022 03:54:50 -0500 Received: from eggs.gnu.org ([209.51.188.92]:54384) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nOy37-00045i-Hl for qemu-devel@nongnu.org; Tue, 01 Mar 2022 03:40:01 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]:53440) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nOy35-0007ww-Tv for qemu-devel@nongnu.org; Tue, 01 Mar 2022 03:40:01 -0500 Received: from mail-pj1-f72.google.com (mail-pj1-f72.google.com [209.85.216.72]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-643-kpQYKNC0MIqnop3hHCszqA-1; Tue, 01 Mar 2022 03:39:58 -0500 Received: by mail-pj1-f72.google.com with SMTP id m3-20020a17090a3f8300b001bd33af55f2so1211265pjc.1 for ; Tue, 01 Mar 2022 00:39:58 -0800 (PST) Received: from localhost.localdomain ([94.177.118.144]) by smtp.gmail.com with ESMTPSA id l1-20020a17090aec0100b001bc6d8bb27dsm1439987pjy.37.2022.03.01.00.39.54 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Tue, 01 Mar 2022 00:39:56 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1646123999; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=xEwh2yYCSs1yyakfukGjUpFviOTSZCevdwq6ZVOn9N8=; b=XI7v3sG9dmONhHqmkzQ3CW/vmawtgD90nlxg4NKON5TocOHbd0G5TfAlDgfayaYv4b1b1Y 6MdSbvbaCbW9tikGvPKUbyg8CyOTHztIh2W1h5Y726AP4uXD7YX3jvqt+dUeNml8i+i/RI 7jjvxEJyKXEDshXpJ5tUDzbydopeJnE= X-MC-Unique: kpQYKNC0MIqnop3hHCszqA-1 X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=xEwh2yYCSs1yyakfukGjUpFviOTSZCevdwq6ZVOn9N8=; b=nMdl2hflZiofUVZNiTxDvSp2pW4blF0YFllaeb1lw1+aL0NQ2eRwU65zE1Jgur84KA 3xUncNjCQaVs9Yi50g/BTQdu1zHgtSx35V8mLlJS479YGehVwABYpocj/YgfYjASoIfI jmIAORvQS5bvxoy8/ufcIIsTtrrNopo2vZe//k4E6OiLAjUfLE7n8saWXIKrDxX+/7AZ u/mHln/QEDujoe0jqydYaGodh+dDkP6h/epuyYviFErhs4qx9mNkq5VuM8vRIU1praJV TDuYsGF/t9QB9K7QP6ub2H3mxK2xy+hVhAHwXuFdMXEZOaeDUDobrXhCqmmrF9xDHanL FgOg== X-Gm-Message-State: AOAM531SCklrmhWUHat/dSUvtQbKOhvtVTxkFFbtrONyRPhd9Ys2HmAO mlWtNLeowm3xT9Pddz3Lai1IEGEVatTvQO3g9FgU6zWI8mF1gxqI5s5ih1rZ6IVeoVHFjl+aoyY eBY4f59AA3pxWEq31nUJKrXUV6W/5JglcLAOv9F2i0Qsj8O/D2N5mH4NtOInc18zw X-Received: by 2002:a63:4a4c:0:b0:375:6898:f6b6 with SMTP id j12-20020a634a4c000000b003756898f6b6mr20667064pgl.83.1646123996990; Tue, 01 Mar 2022 00:39:56 -0800 (PST) X-Google-Smtp-Source: ABdhPJw0LsGwMw8m9Ye9HNg7Y3Bg/woKeQ5x+KXV2boYQSCebqy7FPqNkOfU1zZvIcPig/PJ+mSHGw== X-Received: by 2002:a63:4a4c:0:b0:375:6898:f6b6 with SMTP id j12-20020a634a4c000000b003756898f6b6mr20667040pgl.83.1646123996595; Tue, 01 Mar 2022 00:39:56 -0800 (PST) From: Peter Xu To: qemu-devel@nongnu.org Subject: [PATCH v2 08/25] migration: Add pss.postcopy_requested status Date: Tue, 1 Mar 2022 16:39:08 +0800 Message-Id: <20220301083925.33483-9-peterx@redhat.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20220301083925.33483-1-peterx@redhat.com> References: <20220301083925.33483-1-peterx@redhat.com> MIME-Version: 1.0 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=peterx@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=peterx@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -28 X-Spam_score: -2.9 X-Spam_bar: -- X-Spam_report: (-2.9 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.082, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H5=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Juan Quintela , "Dr . David Alan Gilbert" , peterx@redhat.com, Leonardo Bras Soares Passos Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZM-MESSAGEID: 1646124893040100001 Content-Type: text/plain; charset="utf-8" This boolean flag shows whether the current page during migration is trigge= red by postcopy or not. Then in ram_save_host_page() and deeper stack we'll be able to have a reference on the priority of this page. Reviewed-by: Dr. David Alan Gilbert Signed-off-by: Peter Xu --- migration/ram.c | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/migration/ram.c b/migration/ram.c index 9516dd655a..f1de1a06e4 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -414,6 +414,8 @@ struct PageSearchStatus { unsigned long page; /* Set once we wrap around */ bool complete_round; + /* Whether current page is explicitly requested by postcopy */ + bool postcopy_requested; }; typedef struct PageSearchStatus PageSearchStatus; =20 @@ -1487,6 +1489,9 @@ retry: */ static bool find_dirty_block(RAMState *rs, PageSearchStatus *pss, bool *ag= ain) { + /* This is not a postcopy requested page */ + pss->postcopy_requested =3D false; + pss->page =3D migration_bitmap_find_dirty(rs, pss->block, pss->page); if (pss->complete_round && pss->block =3D=3D rs->last_seen_block && pss->page >=3D rs->last_page) { @@ -1981,6 +1986,7 @@ static bool get_queued_page(RAMState *rs, PageSearchS= tatus *pss) * really rare. */ pss->complete_round =3D false; + pss->postcopy_requested =3D true; } =20 return !!block; --=20 2.32.0 From nobody Mon Feb 9 00:30:16 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 164612532418899.9373488756446; Tue, 1 Mar 2022 01:02:04 -0800 (PST) Received: from localhost ([::1]:37136 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nOyOR-0007gc-A6 for importer@patchew.org; Tue, 01 Mar 2022 04:02:03 -0500 Received: from eggs.gnu.org ([209.51.188.92]:54464) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nOy3A-00048A-Li for qemu-devel@nongnu.org; Tue, 01 Mar 2022 03:40:04 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:42352) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nOy38-0007y3-SX for qemu-devel@nongnu.org; Tue, 01 Mar 2022 03:40:04 -0500 Received: from mail-pl1-f198.google.com (mail-pl1-f198.google.com [209.85.214.198]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-204-B-r03E_HO62t223aYsJysw-1; Tue, 01 Mar 2022 03:40:01 -0500 Received: by mail-pl1-f198.google.com with SMTP id w13-20020a1709027b8d00b0014fb4f012d3so5939944pll.12 for ; Tue, 01 Mar 2022 00:40:01 -0800 (PST) Received: from localhost.localdomain ([94.177.118.144]) by smtp.gmail.com with ESMTPSA id l1-20020a17090aec0100b001bc6d8bb27dsm1439987pjy.37.2022.03.01.00.39.57 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Tue, 01 Mar 2022 00:39:59 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1646124002; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=ZXUvDOAZVCZx2PDK+m45rb6PvTxxwhcJ0/eKmX4lZqw=; b=SUSCRc4Ajntj6sYUO9avOuX1sJIXJOQRltgEoF8QBYLDS8KR+3xOUNFvkzH7/ZzqjGuBSd F9xe6BWTKFNkb5QTcG4EHf31nxcA6BU+eNpSJEQV2PScVquW7MvGre8hfJyUW4O47cn4Ue 5M10sHvJuzZF2Za9bWgoD5iQGKwWGFM= X-MC-Unique: B-r03E_HO62t223aYsJysw-1 X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=ZXUvDOAZVCZx2PDK+m45rb6PvTxxwhcJ0/eKmX4lZqw=; b=rPFjNtUrXu6MDPKUtwRRVklriuUKYNV139zCspQE5njLqzCezTaT9GtDtHXql6VVW0 HppS6qhNszyLqK4V/qQMKKEER0/+sVCVBVkbroWV4Bsj70AMHv+yG54bwf9JcPWX3PhT 7J0r5FtJmZCOg+eQMWAeytj0fK+J9gm1UVgcX65irZzIMOFakQhBqoBeurwUei1Ufn5W t5Ti7jnD/XTCVnXxt8Ozhq0Os0fLhiSpHH2vCnusN13luDP+4ar7FBZoQr9wqGUzcjJF kwG1lOjeOVIawvwjhPmbPI5TcIs3UrJAhRW8rxz99HeYHDWaT2oNf3fdB4pwGlg89PCk dl9Q== X-Gm-Message-State: AOAM5334jQuM0eJ8kLvEFBLozBab7TjaL0KLRZiTv+zhZlvKuq8AwLH/ 7rJM+zAiWCFY/fPV6iGN7FOkj4z2oixSiU4z3tjtQSe1u742bOksKffpU4pOBJQPwGF22svzPRP 3kaz+mhegFe83i4Vey0FzYaSQJ2MMzQgOJMZ/Cq35XA55OLN/t5Uf8rSg8gyxMGkr X-Received: by 2002:a63:6442:0:b0:362:ad55:f5e5 with SMTP id y63-20020a636442000000b00362ad55f5e5mr20533626pgb.180.1646123999891; Tue, 01 Mar 2022 00:39:59 -0800 (PST) X-Google-Smtp-Source: ABdhPJx+XmbDpBTAO+fNhOZUfh7tdYxE0EFdQ/qqLtIuvt5J153NKwwbmf3Zs4wou3RQsei0UMlcnA== X-Received: by 2002:a63:6442:0:b0:362:ad55:f5e5 with SMTP id y63-20020a636442000000b00362ad55f5e5mr20533601pgb.180.1646123999488; Tue, 01 Mar 2022 00:39:59 -0800 (PST) From: Peter Xu To: qemu-devel@nongnu.org Subject: [PATCH v2 09/25] migration: Move migrate_allow_multifd and helpers into migration.c Date: Tue, 1 Mar 2022 16:39:09 +0800 Message-Id: <20220301083925.33483-10-peterx@redhat.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20220301083925.33483-1-peterx@redhat.com> References: <20220301083925.33483-1-peterx@redhat.com> MIME-Version: 1.0 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=peterx@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=peterx@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -28 X-Spam_score: -2.9 X-Spam_bar: -- X-Spam_report: (-2.9 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.082, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H5=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Juan Quintela , "Dr . David Alan Gilbert" , peterx@redhat.com, Leonardo Bras Soares Passos Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZM-MESSAGEID: 1646125325441100001 Content-Type: text/plain; charset="utf-8" This variable, along with its helpers, is used to detect whether multiple channel will be supported for migration. In follow up patches, there'll be other capability that requires multi-channels. Hence move it outside multi= fd specific code and make it public. Meanwhile rename it from "multifd" to "multi_channels" to show its real meaning. Reviewed-by: Dr. David Alan Gilbert Signed-off-by: Peter Xu --- migration/migration.c | 22 +++++++++++++++++----- migration/migration.h | 3 +++ migration/multifd.c | 19 ++++--------------- migration/multifd.h | 2 -- 4 files changed, 24 insertions(+), 22 deletions(-) diff --git a/migration/migration.c b/migration/migration.c index bcc385b94b..6e4cc9cc87 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -180,6 +180,18 @@ static int migration_maybe_pause(MigrationState *s, int new_state); static void migrate_fd_cancel(MigrationState *s); =20 +static bool migrate_allow_multi_channels =3D true; + +void migrate_protocol_allow_multi_channels(bool allow) +{ + migrate_allow_multi_channels =3D allow; +} + +bool migrate_multi_channels_is_allowed(void) +{ + return migrate_allow_multi_channels; +} + static gint page_request_addr_cmp(gconstpointer ap, gconstpointer bp) { uintptr_t a =3D (uintptr_t) ap, b =3D (uintptr_t) bp; @@ -463,12 +475,12 @@ static void qemu_start_incoming_migration(const char = *uri, Error **errp) { const char *p =3D NULL; =20 - migrate_protocol_allow_multifd(false); /* reset it anyway */ + migrate_protocol_allow_multi_channels(false); /* reset it anyway */ qapi_event_send_migration(MIGRATION_STATUS_SETUP); if (strstart(uri, "tcp:", &p) || strstart(uri, "unix:", NULL) || strstart(uri, "vsock:", NULL)) { - migrate_protocol_allow_multifd(true); + migrate_protocol_allow_multi_channels(true); socket_start_incoming_migration(p ? p : uri, errp); #ifdef CONFIG_RDMA } else if (strstart(uri, "rdma:", &p)) { @@ -1255,7 +1267,7 @@ static bool migrate_caps_check(bool *cap_list, =20 /* incoming side only */ if (runstate_check(RUN_STATE_INMIGRATE) && - !migrate_multifd_is_allowed() && + !migrate_multi_channels_is_allowed() && cap_list[MIGRATION_CAPABILITY_MULTIFD]) { error_setg(errp, "multifd is not supported by current protocol"); return false; @@ -2313,11 +2325,11 @@ void qmp_migrate(const char *uri, bool has_blk, boo= l blk, } } =20 - migrate_protocol_allow_multifd(false); + migrate_protocol_allow_multi_channels(false); if (strstart(uri, "tcp:", &p) || strstart(uri, "unix:", NULL) || strstart(uri, "vsock:", NULL)) { - migrate_protocol_allow_multifd(true); + migrate_protocol_allow_multi_channels(true); socket_start_outgoing_migration(s, p ? p : uri, &local_err); #ifdef CONFIG_RDMA } else if (strstart(uri, "rdma:", &p)) { diff --git a/migration/migration.h b/migration/migration.h index d8b9850eae..d677a750c9 100644 --- a/migration/migration.h +++ b/migration/migration.h @@ -429,4 +429,7 @@ void migration_cancel(const Error *error); void populate_vfio_info(MigrationInfo *info); void postcopy_temp_page_reset(PostcopyTmpPage *tmp_page); =20 +bool migrate_multi_channels_is_allowed(void); +void migrate_protocol_allow_multi_channels(bool allow); + #endif diff --git a/migration/multifd.c b/migration/multifd.c index 76b57a7177..180586dcde 100644 --- a/migration/multifd.c +++ b/migration/multifd.c @@ -517,7 +517,7 @@ void multifd_save_cleanup(void) { int i; =20 - if (!migrate_use_multifd() || !migrate_multifd_is_allowed()) { + if (!migrate_use_multifd() || !migrate_multi_channels_is_allowed()) { return; } multifd_send_terminate_threads(NULL); @@ -858,17 +858,6 @@ cleanup: multifd_new_send_channel_cleanup(p, sioc, local_err); } =20 -static bool migrate_allow_multifd =3D true; -void migrate_protocol_allow_multifd(bool allow) -{ - migrate_allow_multifd =3D allow; -} - -bool migrate_multifd_is_allowed(void) -{ - return migrate_allow_multifd; -} - int multifd_save_setup(Error **errp) { int thread_count; @@ -879,7 +868,7 @@ int multifd_save_setup(Error **errp) if (!migrate_use_multifd()) { return 0; } - if (!migrate_multifd_is_allowed()) { + if (!migrate_multi_channels_is_allowed()) { error_setg(errp, "multifd is not supported by current protocol"); return -1; } @@ -980,7 +969,7 @@ int multifd_load_cleanup(Error **errp) { int i; =20 - if (!migrate_use_multifd() || !migrate_multifd_is_allowed()) { + if (!migrate_use_multifd() || !migrate_multi_channels_is_allowed()) { return 0; } multifd_recv_terminate_threads(NULL); @@ -1129,7 +1118,7 @@ int multifd_load_setup(Error **errp) if (!migrate_use_multifd()) { return 0; } - if (!migrate_multifd_is_allowed()) { + if (!migrate_multi_channels_is_allowed()) { error_setg(errp, "multifd is not supported by current protocol"); return -1; } diff --git a/migration/multifd.h b/migration/multifd.h index 4dda900a0b..636e599395 100644 --- a/migration/multifd.h +++ b/migration/multifd.h @@ -13,8 +13,6 @@ #ifndef QEMU_MIGRATION_MULTIFD_H #define QEMU_MIGRATION_MULTIFD_H =20 -bool migrate_multifd_is_allowed(void); -void migrate_protocol_allow_multifd(bool allow); int multifd_save_setup(Error **errp); void multifd_save_cleanup(void); int multifd_load_setup(Error **errp); --=20 2.32.0 From nobody Mon Feb 9 00:30:16 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1646124899267902.2627740984346; Tue, 1 Mar 2022 00:54:59 -0800 (PST) Received: from localhost ([::1]:56744 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nOyHZ-0001fe-UT for importer@patchew.org; Tue, 01 Mar 2022 03:54:57 -0500 Received: from eggs.gnu.org ([209.51.188.92]:54488) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nOy3C-00049i-Qw for qemu-devel@nongnu.org; Tue, 01 Mar 2022 03:40:09 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:51379) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nOy3B-000891-7s for qemu-devel@nongnu.org; Tue, 01 Mar 2022 03:40:06 -0500 Received: from mail-pf1-f200.google.com (mail-pf1-f200.google.com [209.85.210.200]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-606-tMsVCqWZN8K49j0AofARaA-1; Tue, 01 Mar 2022 03:40:03 -0500 Received: by mail-pf1-f200.google.com with SMTP id v127-20020a622f85000000b004f3dfd386e8so6375328pfv.16 for ; Tue, 01 Mar 2022 00:40:03 -0800 (PST) Received: from localhost.localdomain ([94.177.118.144]) by smtp.gmail.com with ESMTPSA id l1-20020a17090aec0100b001bc6d8bb27dsm1439987pjy.37.2022.03.01.00.39.59 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Tue, 01 Mar 2022 00:40:01 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1646124004; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=41Js5PNU7Y+xQ6FoNmu0DLJBoBEaZC2FdumT8DVak1w=; b=dtKrRoBoE4r6iHrt1yW5PxnlkwEj7RFykaQzPOioee0O4VgS2zUmNgNm1WLZty912TYywK ghzqu2kHHR0f61ShDYBT1E/Dyjp5U4jYvTyO6MfnSfBTNWGgPaJAYKSYo7Clazn1kgsfjK s8zeJJ3g/Btr8IsMjyHbwqr6G27+CqU= X-MC-Unique: tMsVCqWZN8K49j0AofARaA-1 X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=41Js5PNU7Y+xQ6FoNmu0DLJBoBEaZC2FdumT8DVak1w=; b=obaOj/DIU0XPqDUDCaYV4KGyA0u5enxQ8f+AqPvGKB4+GuRhgs+MzCVmiT2qmiUrbZ VttgTbx8lU9ma06M7oWQ4S+XetinT1bakdNpakKJoPvNVcKN1blTadyx40maTdWXW/i5 JVrrqm2QzOVSgDiWKe03jiYtDYxWV0b33n1oUGhZSBb2q03PG11/qqSf2Sy5grWHAlA8 KmUDh2qurCADk+V5IETUN5LNdtewZf/b552mw1jA0mvyv+Q++7e1j1ckGhwezyvOa2g9 /Vs64rL/VyIIu6J86ssNFptcvseJa6ZTAMX3kwrj1EjKz0bhAWmqWiWIH8SLzcMj11nN FdVA== X-Gm-Message-State: AOAM531WjrxNe/qB/VgxZukS0ai4qZDnKqjNP730+WQoTBpR2tDgfQ+2 P4FPAmTbPDTC8w5TIdj8Nwq2ub0glrhnRt0QQrOPa3o448vniAYWf1DW/MMCTPkNys95NHbIoug 5KF+HFM5YtYAhi+Gv4ZfFdCCugB2ZetR3iVc2qYHChsGrapGFPz2uGyfbXfCND8ct X-Received: by 2002:a63:70b:0:b0:373:9e98:d8f9 with SMTP id 11-20020a63070b000000b003739e98d8f9mr20368703pgh.242.1646124002501; Tue, 01 Mar 2022 00:40:02 -0800 (PST) X-Google-Smtp-Source: ABdhPJzdfbtVKd2tQsbkxJxV6Rp4kXkM8GK07qN7v4dB7ZxaGrMLpe+q3+bjzzrUAGJCxJqgWxxxCw== X-Received: by 2002:a63:70b:0:b0:373:9e98:d8f9 with SMTP id 11-20020a63070b000000b003739e98d8f9mr20368677pgh.242.1646124002131; Tue, 01 Mar 2022 00:40:02 -0800 (PST) From: Peter Xu To: qemu-devel@nongnu.org Subject: [PATCH v2 10/25] migration: Enlarge postcopy recovery to capture !-EIO too Date: Tue, 1 Mar 2022 16:39:10 +0800 Message-Id: <20220301083925.33483-11-peterx@redhat.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20220301083925.33483-1-peterx@redhat.com> References: <20220301083925.33483-1-peterx@redhat.com> MIME-Version: 1.0 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=peterx@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=peterx@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -28 X-Spam_score: -2.9 X-Spam_bar: -- X-Spam_report: (-2.9 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.082, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H5=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Juan Quintela , "Dr . David Alan Gilbert" , peterx@redhat.com, Leonardo Bras Soares Passos Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZM-MESSAGEID: 1646124901248100001 Content-Type: text/plain; charset="utf-8" We used to have quite a few places making sure -EIO happened and that's the only way to trigger postcopy recovery. That's based on the assumption that we'll only return -EIO for channel issues. It'll work in 99.99% cases but logically that won't cover some corner cases. One example is e.g. ram_block_from_stream() could fail with an interrupted network, then -EINVAL will be returned instead of -EIO. I remembered Dave Gilbert pointed that out before, but somehow this is overlooked. Neither did I encounter anything outside the -EIO error. However we'd better touch that up before it triggers a rare VM data loss du= ring live migrating. To cover as much those cases as possible, remove the -EIO restriction on triggering the postcopy recovery, because even if it's not a channel failur= e, we can't do anything better than halting QEMU anyway - the corpse of the process may even be used by a good hand to dig out useful memory regions, or the admin could simply kill the process later on. Reviewed-by: Dr. David Alan Gilbert Signed-off-by: Peter Xu --- migration/migration.c | 4 ++-- migration/postcopy-ram.c | 2 +- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/migration/migration.c b/migration/migration.c index 6e4cc9cc87..67520d3105 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -2877,7 +2877,7 @@ retry: out: res =3D qemu_file_get_error(rp); if (res) { - if (res =3D=3D -EIO && migration_in_postcopy()) { + if (res && migration_in_postcopy()) { /* * Maybe there is something we can do: it looks like a * network down issue, and we pause for a recovery. @@ -3478,7 +3478,7 @@ static MigThrError migration_detect_error(MigrationSt= ate *s) error_free(local_error); } =20 - if (state =3D=3D MIGRATION_STATUS_POSTCOPY_ACTIVE && ret =3D=3D -EIO) { + if (state =3D=3D MIGRATION_STATUS_POSTCOPY_ACTIVE && ret) { /* * For postcopy, we allow the network to be down for a * while. After that, it can be continued by a diff --git a/migration/postcopy-ram.c b/migration/postcopy-ram.c index d08d396c63..b0d12d5053 100644 --- a/migration/postcopy-ram.c +++ b/migration/postcopy-ram.c @@ -1039,7 +1039,7 @@ retry: msg.arg.pagefault.address); if (ret) { /* May be network failure, try to wait for recovery */ - if (ret =3D=3D -EIO && postcopy_pause_fault_thread(mis)) { + if (postcopy_pause_fault_thread(mis)) { /* We got reconnected somehow, try to continue */ goto retry; } else { --=20 2.32.0 From nobody Mon Feb 9 00:30:16 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1646125737084678.8481424381094; Tue, 1 Mar 2022 01:08:57 -0800 (PST) Received: from localhost ([::1]:45458 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nOyV5-0005Ju-P8 for importer@patchew.org; Tue, 01 Mar 2022 04:08:55 -0500 Received: from eggs.gnu.org ([209.51.188.92]:54566) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nOy3G-0004Ae-6I for qemu-devel@nongnu.org; Tue, 01 Mar 2022 03:40:10 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:58924) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nOy3D-0008AG-Vj for qemu-devel@nongnu.org; Tue, 01 Mar 2022 03:40:09 -0500 Received: from mail-pf1-f199.google.com (mail-pf1-f199.google.com [209.85.210.199]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-612-R8dhXbwnPwGi-q_PXqkDKg-1; Tue, 01 Mar 2022 03:40:06 -0500 Received: by mail-pf1-f199.google.com with SMTP id y28-20020aa793dc000000b004e160274e3eso9354935pff.18 for ; Tue, 01 Mar 2022 00:40:06 -0800 (PST) Received: from localhost.localdomain ([94.177.118.144]) by smtp.gmail.com with ESMTPSA id l1-20020a17090aec0100b001bc6d8bb27dsm1439987pjy.37.2022.03.01.00.40.02 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Tue, 01 Mar 2022 00:40:04 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1646124007; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=pXE1AeYTmM/PzAdQpdkXZXkqq6LdQ92xhjUt8UsD2ME=; b=WCtfTuiFWm0ir+oa+Ahl9PnDAFW1Zb+NMmctwQhysP3rpDc/7bgVBzJUR4CiCiLNlCOS1H FDFncYByInRsWgjozn9snJI3InDqsukJ9D2YrEEEFx7HUlgjs6JMlW3JXznKcl4puItcPP FePsYYvg5/t8xD76h92UAKtC0MNP+Gg= X-MC-Unique: R8dhXbwnPwGi-q_PXqkDKg-1 X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=pXE1AeYTmM/PzAdQpdkXZXkqq6LdQ92xhjUt8UsD2ME=; b=yI0kml3YMWFb0iuOyk02yXmiGw/Uerlleuc4IHaH/yLbjyyxniIgpgMEWjPrekAJdr H9NIPmUzxmyIs8jnsqE5qgIsXQAjBFY2SfHQx485pHt8VMIkjoqmq0gMNdCXPh3OErk6 RLVqtjC4wxXBHGSumraP1fwLdZ8Ce3mk00ua8pYZ9/zU5TC3XHyRKY6SlAdAyZdSIr4e cRUOztp0B/YalccLTbA7Oztl2fCT33Qhzhu0J1U89i2E0bloazIJipNCFiAwBAUE7Pwu KLEWUio8mXNjD8PxiAB2ydutjTb+oEpU3jSHiWDxKALWRrvxSyBpEunQ4mSkdTDlq+7F ADQw== X-Gm-Message-State: AOAM531PMcl2a91J9WoXZOB/2davws8EUzLVyC9qJiMllhMFYz56ObPU 2/VPGrzmkCeiUrbcxYNQxn0sPr3Ue81E3x8T0N77xi2c6Ga8ZeVE+Fc9wI/ynseZgVKBHkZEWhX JwgWI2wD7/KHKM/KdaPybrdoeUwEWdfNl7owGY8aMWlK4XkOZjDvqZiEz+DC7Qq64 X-Received: by 2002:a17:90b:3145:b0:1bc:5855:f94d with SMTP id ip5-20020a17090b314500b001bc5855f94dmr20785248pjb.55.1646124005359; Tue, 01 Mar 2022 00:40:05 -0800 (PST) X-Google-Smtp-Source: ABdhPJwXKBaLRpHuO7IDwdNyjtOJiibux/w8EtgqGs5ICi3m7nAgLrgdvlVs0F2pQvm7FqJXkx1Qng== X-Received: by 2002:a17:90b:3145:b0:1bc:5855:f94d with SMTP id ip5-20020a17090b314500b001bc5855f94dmr20785219pjb.55.1646124004994; Tue, 01 Mar 2022 00:40:04 -0800 (PST) From: Peter Xu To: qemu-devel@nongnu.org Subject: [PATCH v2 11/25] migration: postcopy_pause_fault_thread() never fails Date: Tue, 1 Mar 2022 16:39:11 +0800 Message-Id: <20220301083925.33483-12-peterx@redhat.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20220301083925.33483-1-peterx@redhat.com> References: <20220301083925.33483-1-peterx@redhat.com> MIME-Version: 1.0 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=peterx@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=peterx@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -28 X-Spam_score: -2.9 X-Spam_bar: -- X-Spam_report: (-2.9 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.082, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H5=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Juan Quintela , "Dr . David Alan Gilbert" , peterx@redhat.com, Leonardo Bras Soares Passos Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZM-MESSAGEID: 1646125738896100001 Content-Type: text/plain; charset="utf-8" Per the title, remove the return code and simplify the callers as the errors will never be triggered. No functional change intended. Reviewed-by: Dr. David Alan Gilbert Signed-off-by: Peter Xu --- migration/postcopy-ram.c | 25 ++++--------------------- 1 file changed, 4 insertions(+), 21 deletions(-) diff --git a/migration/postcopy-ram.c b/migration/postcopy-ram.c index b0d12d5053..32c52f4b1d 100644 --- a/migration/postcopy-ram.c +++ b/migration/postcopy-ram.c @@ -891,15 +891,11 @@ static void mark_postcopy_blocktime_end(uintptr_t add= r) affected_cpu); } =20 -static bool postcopy_pause_fault_thread(MigrationIncomingState *mis) +static void postcopy_pause_fault_thread(MigrationIncomingState *mis) { trace_postcopy_pause_fault_thread(); - qemu_sem_wait(&mis->postcopy_pause_sem_fault); - trace_postcopy_pause_fault_thread_continued(); - - return true; } =20 /* @@ -959,13 +955,7 @@ static void *postcopy_ram_fault_thread(void *opaque) * broken already using the event. We should hold until * the channel is rebuilt. */ - if (postcopy_pause_fault_thread(mis)) { - /* Continue to read the userfaultfd */ - } else { - error_report("%s: paused but don't allow to continue", - __func__); - break; - } + postcopy_pause_fault_thread(mis); } =20 if (pfd[1].revents) { @@ -1039,15 +1029,8 @@ retry: msg.arg.pagefault.address); if (ret) { /* May be network failure, try to wait for recovery */ - if (postcopy_pause_fault_thread(mis)) { - /* We got reconnected somehow, try to continue */ - goto retry; - } else { - /* This is a unavoidable fault */ - error_report("%s: postcopy_request_page() get %d", - __func__, ret); - break; - } + postcopy_pause_fault_thread(mis); + goto retry; } } =20 --=20 2.32.0 From nobody Mon Feb 9 00:30:16 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1646124680849351.44000832078905; Tue, 1 Mar 2022 00:51:20 -0800 (PST) Received: from localhost ([::1]:49948 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nOyE4-0005N6-87 for importer@patchew.org; Tue, 01 Mar 2022 03:51:20 -0500 Received: from eggs.gnu.org ([209.51.188.92]:54582) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nOy3H-0004EZ-PV for qemu-devel@nongnu.org; Tue, 01 Mar 2022 03:40:12 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]:24225) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nOy3G-0008Aj-9Y for qemu-devel@nongnu.org; Tue, 01 Mar 2022 03:40:11 -0500 Received: from mail-pg1-f200.google.com (mail-pg1-f200.google.com [209.85.215.200]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-591-_5nI2PkZOYeEUtEaX5faiw-1; Tue, 01 Mar 2022 03:40:09 -0500 Received: by mail-pg1-f200.google.com with SMTP id a12-20020a65640c000000b003756296df5cso8168127pgv.19 for ; Tue, 01 Mar 2022 00:40:08 -0800 (PST) Received: from localhost.localdomain ([94.177.118.144]) by smtp.gmail.com with ESMTPSA id l1-20020a17090aec0100b001bc6d8bb27dsm1439987pjy.37.2022.03.01.00.40.05 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Tue, 01 Mar 2022 00:40:07 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1646124009; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=h8U+0chEqOoO2WXEq/PMFQNiVpucDS0U2rAdxkt/8Jg=; b=C6VnYjD+V9cvsPY7KJcd3+CfWQNlsSbyCZiDaAe8itns22sjstbZfCxBnJJlHdGzVqp76E OGYe1l1NPyRsOOmEBkYE0ZnbjfEotpAMb4TQGoTPedfxaL0wlHqhHTjIYXCHKlRaQHNlDO a79zUii51JHNlEx9Xqb/vi4KG8mouvs= X-MC-Unique: _5nI2PkZOYeEUtEaX5faiw-1 X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=h8U+0chEqOoO2WXEq/PMFQNiVpucDS0U2rAdxkt/8Jg=; b=ETavxuMsUzaev9+xt5pjQfCbULN6oUme8Nb7ibpXvvVz+PgznLE5zlmwbJrEJfJD2Y 1PVLtYIbdY9rhOVN8Z9VAyeH/mVuPaJ3lkjdbjWFpylnicyyT6t1viAxVa6LLebu4AM/ TMgwidUrAuaACmmsQJieWe06N384rWPb2e7jXegOsGRpVkeB2KuSz5rqZbqOLzpl96vU AONcSi7m8IcAb2saw5B3w4PmHdmN+nqU5AgR05iZHfuvJyakivqbC/FhKZKEhCjgJgCU ncaCuFh7dDh6HUkNBFaWv73CEz1ztHShOW0eWUz3CwVSxqJL+3ctE9qpN9DMNGkwyj4/ wpIg== X-Gm-Message-State: AOAM532sl82N/vMNGCrlnZUXw8V2FR12zjxwGEgM1S8hhuAAv/roGGBa 40yJplKPFJQtmUGvzUoiDODEIEGiO6Loe7Ej8OgHgYZSgqvOE5YNcUtO/WzZZ++YWnJXxqJbkf4 1TP/M1JVPjguogu7fiXOyZUYJ6e2FehJv1KTm5hCfGK8U5kuIi4YHlIu4CJlA/Wvx X-Received: by 2002:a63:e758:0:b0:378:8511:cfe7 with SMTP id j24-20020a63e758000000b003788511cfe7mr9866199pgk.126.1646124007843; Tue, 01 Mar 2022 00:40:07 -0800 (PST) X-Google-Smtp-Source: ABdhPJxL9H0eXhDUAcQ89AwZExrMzUTGCge2rDoPYg2aJzMRHrZLzLSjVcLiUedxXc879M2IDaOt0A== X-Received: by 2002:a63:e758:0:b0:378:8511:cfe7 with SMTP id j24-20020a63e758000000b003788511cfe7mr9866180pgk.126.1646124007560; Tue, 01 Mar 2022 00:40:07 -0800 (PST) From: Peter Xu To: qemu-devel@nongnu.org Subject: [PATCH v2 12/25] migration: Export ram_load_postcopy() Date: Tue, 1 Mar 2022 16:39:12 +0800 Message-Id: <20220301083925.33483-13-peterx@redhat.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20220301083925.33483-1-peterx@redhat.com> References: <20220301083925.33483-1-peterx@redhat.com> MIME-Version: 1.0 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=peterx@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=peterx@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -28 X-Spam_score: -2.9 X-Spam_bar: -- X-Spam_report: (-2.9 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.082, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H5=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Juan Quintela , "Dr . David Alan Gilbert" , peterx@redhat.com, Leonardo Bras Soares Passos Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZM-MESSAGEID: 1646124708325100001 Content-Type: text/plain; charset="utf-8" Will be reused in postcopy fast load thread. Reviewed-by: Dr. David Alan Gilbert Signed-off-by: Peter Xu --- migration/ram.c | 2 +- migration/ram.h | 1 + 2 files changed, 2 insertions(+), 1 deletion(-) diff --git a/migration/ram.c b/migration/ram.c index f1de1a06e4..5cb5dfc2cc 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -3645,7 +3645,7 @@ int ram_postcopy_incoming_init(MigrationIncomingState= *mis) * * @f: QEMUFile where to send the data */ -static int ram_load_postcopy(QEMUFile *f) +int ram_load_postcopy(QEMUFile *f) { int flags =3D 0, ret =3D 0; bool place_needed =3D false; diff --git a/migration/ram.h b/migration/ram.h index 2c6dc3675d..ded0a3a086 100644 --- a/migration/ram.h +++ b/migration/ram.h @@ -61,6 +61,7 @@ void ram_postcopy_send_discard_bitmap(MigrationState *ms); /* For incoming postcopy discard */ int ram_discard_range(const char *block_name, uint64_t start, size_t lengt= h); int ram_postcopy_incoming_init(MigrationIncomingState *mis); +int ram_load_postcopy(QEMUFile *f); =20 void ram_handle_compressed(void *host, uint8_t ch, uint64_t size); =20 --=20 2.32.0 From nobody Mon Feb 9 00:30:16 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1646125322999539.6056515124164; Tue, 1 Mar 2022 01:02:02 -0800 (PST) Received: from localhost ([::1]:37122 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nOyOP-0007g7-Ho for importer@patchew.org; Tue, 01 Mar 2022 04:02:01 -0500 Received: from eggs.gnu.org ([209.51.188.92]:54648) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nOy3L-0004F6-RG for qemu-devel@nongnu.org; Tue, 01 Mar 2022 03:40:20 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]:51922) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nOy3J-0008Ca-GV for qemu-devel@nongnu.org; Tue, 01 Mar 2022 03:40:14 -0500 Received: from mail-pj1-f72.google.com (mail-pj1-f72.google.com [209.85.216.72]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-669-O-MkCPZqP82XlNnBU6e33w-1; Tue, 01 Mar 2022 03:40:12 -0500 Received: by mail-pj1-f72.google.com with SMTP id j22-20020a17090aeb1600b001bc32977e07so1055439pjz.7 for ; Tue, 01 Mar 2022 00:40:11 -0800 (PST) Received: from localhost.localdomain ([94.177.118.144]) by smtp.gmail.com with ESMTPSA id l1-20020a17090aec0100b001bc6d8bb27dsm1439987pjy.37.2022.03.01.00.40.07 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Tue, 01 Mar 2022 00:40:10 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1646124012; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=ioPo/zq9TaQigYCLhmNX8dwsBWn+7+sCdTwob7NMCP8=; b=W68wJ0Qpuo/8gROwekATurEcwYRe4WOkwyCEtLk0RF9s+eBp8WQmTWn16xJMIoOTPrJsrt zkoBahw039jpwt5Yz4RaoItzB0vj0hHPe75uWvWu4554/+cTa4xcUa6qT4tQV4qRNzkomY 331KxjgUrdPletpTvZ26k96bkIliHwo= X-MC-Unique: O-MkCPZqP82XlNnBU6e33w-1 X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=ioPo/zq9TaQigYCLhmNX8dwsBWn+7+sCdTwob7NMCP8=; b=h8lJs4GZxZGyUds6xPXK8SjRmguFxj338ADcqNxtLABmHGI7AneJFqlQ39R2nZqCUn aG7JJBzLEoLI55bycqTKiw6NDOpT2kIronC0apHyJUnc3ws1Ax47zQ9gEyznv4s5/LO4 SKPr6En9ComdpzwqfUnT4BzFv5tmh2S1K8PvgtKdtWaYO6EyKhgShw+JwZoXtu1MuLtP znnXSf06Hw4BTelSSneKIBgs7QdEuXfUBVOdd3j9j9YS+XOx/uJrlcc6b1fglExi7rwa KYyQ/85SMlaxKmUccqBPVaWHsAqoAt8Gz7vjoY1xC3jebgKgLWQFavJSAovDCDpPn8mr qnXA== X-Gm-Message-State: AOAM532q9Po+kovQ0a3JdXawA3TKaxUQZxe2WK83J8LhhMQYzZeYjJRe wGdVZ/l+Nrpzzu5cyyIkTjOG8NcL0I1KOW74a+ztHz2Eni/GJqbj+tqR2kASAW9O3v9FqJnc6rW SZ/c0GdSeHu3VDHY20oeyQManCOEaCHBaqs9bLMLiGA3Q3elYoXwoAWRvlhiML6Hz X-Received: by 2002:a63:1f42:0:b0:378:7fb4:6491 with SMTP id q2-20020a631f42000000b003787fb46491mr10427700pgm.478.1646124010791; Tue, 01 Mar 2022 00:40:10 -0800 (PST) X-Google-Smtp-Source: ABdhPJweRKUqjGDhbe/G2rxVkYPgXhmbGz7fWkMIEfMZa9DCDEBHwlWrbYgiHg7clUi4JBr0BDxzgQ== X-Received: by 2002:a63:1f42:0:b0:378:7fb4:6491 with SMTP id q2-20020a631f42000000b003787fb46491mr10427678pgm.478.1646124010425; Tue, 01 Mar 2022 00:40:10 -0800 (PST) From: Peter Xu To: qemu-devel@nongnu.org Subject: [PATCH v2 13/25] migration: Move channel setup out of postcopy_try_recover() Date: Tue, 1 Mar 2022 16:39:13 +0800 Message-Id: <20220301083925.33483-14-peterx@redhat.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20220301083925.33483-1-peterx@redhat.com> References: <20220301083925.33483-1-peterx@redhat.com> MIME-Version: 1.0 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=peterx@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=peterx@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -28 X-Spam_score: -2.9 X-Spam_bar: -- X-Spam_report: (-2.9 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.082, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H5=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Juan Quintela , "Dr . David Alan Gilbert" , peterx@redhat.com, Leonardo Bras Soares Passos Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZM-MESSAGEID: 1646125323392100003 Content-Type: text/plain; charset="utf-8" We used to use postcopy_try_recover() to replace migration_incoming_setup()= to setup incoming channels. That's fine for the old world, but in the new wor= ld there can be more than one channels that need setup. Better move the chann= el setup out of it so that postcopy_try_recover() only handles the last phase = of switching to the recovery phase. To do that in migration_fd_process_incoming(), move the postcopy_try_recove= r() call to be after migration_incoming_setup(), which will setup the channels. While in migration_ioc_process_incoming(), postpone the recover() routine r= ight before we'll jump into migration_incoming_process(). A side benefit is we don't need to pass in QEMUFile* to postcopy_try_recove= r() anymore. Remove it. Reviewed-by: Dr. David Alan Gilbert Signed-off-by: Peter Xu --- migration/migration.c | 23 +++++++++++------------ 1 file changed, 11 insertions(+), 12 deletions(-) diff --git a/migration/migration.c b/migration/migration.c index 67520d3105..b2e6446457 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -665,19 +665,20 @@ void migration_incoming_process(void) } =20 /* Returns true if recovered from a paused migration, otherwise false */ -static bool postcopy_try_recover(QEMUFile *f) +static bool postcopy_try_recover(void) { MigrationIncomingState *mis =3D migration_incoming_get_current(); =20 if (mis->state =3D=3D MIGRATION_STATUS_POSTCOPY_PAUSED) { /* Resumed from a paused postcopy migration */ =20 - mis->from_src_file =3D f; + /* This should be set already in migration_incoming_setup() */ + assert(mis->from_src_file); /* Postcopy has standalone thread to do vm load */ - qemu_file_set_blocking(f, true); + qemu_file_set_blocking(mis->from_src_file, true); =20 /* Re-configure the return path */ - mis->to_src_file =3D qemu_file_get_return_path(f); + mis->to_src_file =3D qemu_file_get_return_path(mis->from_src_file); =20 migrate_set_state(&mis->state, MIGRATION_STATUS_POSTCOPY_PAUSED, MIGRATION_STATUS_POSTCOPY_RECOVER); @@ -698,11 +699,10 @@ static bool postcopy_try_recover(QEMUFile *f) =20 void migration_fd_process_incoming(QEMUFile *f, Error **errp) { - if (postcopy_try_recover(f)) { + if (!migration_incoming_setup(f, errp)) { return; } - - if (!migration_incoming_setup(f, errp)) { + if (postcopy_try_recover()) { return; } migration_incoming_process(); @@ -718,11 +718,6 @@ void migration_ioc_process_incoming(QIOChannel *ioc, E= rror **errp) /* The first connection (multifd may have multiple) */ QEMUFile *f =3D qemu_fopen_channel_input(ioc); =20 - /* If it's a recovery, we're done */ - if (postcopy_try_recover(f)) { - return; - } - if (!migration_incoming_setup(f, errp)) { return; } @@ -743,6 +738,10 @@ void migration_ioc_process_incoming(QIOChannel *ioc, E= rror **errp) } =20 if (start_migration) { + /* If it's a recovery, we're done */ + if (postcopy_try_recover()) { + return; + } migration_incoming_process(); } } --=20 2.32.0 From nobody Mon Feb 9 00:30:16 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1646125006454442.44129678795616; Tue, 1 Mar 2022 00:56:46 -0800 (PST) Received: from localhost ([::1]:58540 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nOyJJ-0002uR-Ee for importer@patchew.org; Tue, 01 Mar 2022 03:56:45 -0500 Received: from eggs.gnu.org ([209.51.188.92]:54672) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nOy3N-0004FJ-Pn for qemu-devel@nongnu.org; Tue, 01 Mar 2022 03:40:21 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]:49558) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nOy3M-0008DC-6w for qemu-devel@nongnu.org; Tue, 01 Mar 2022 03:40:17 -0500 Received: from mail-pj1-f71.google.com (mail-pj1-f71.google.com [209.85.216.71]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-393-GsDzzPyvM8GwGBifPta5bA-1; Tue, 01 Mar 2022 03:40:14 -0500 Received: by mail-pj1-f71.google.com with SMTP id m3-20020a17090a3f8300b001bd33af55f2so1211714pjc.1 for ; Tue, 01 Mar 2022 00:40:14 -0800 (PST) Received: from localhost.localdomain ([94.177.118.144]) by smtp.gmail.com with ESMTPSA id l1-20020a17090aec0100b001bc6d8bb27dsm1439987pjy.37.2022.03.01.00.40.10 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Tue, 01 Mar 2022 00:40:12 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1646124015; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=mmPTCZCZTk49wlE2CkFHTKouE1bMT/b1++uEYojsNak=; b=bH3O+K10jLOvGBVyqDk8mmaW16mGAwzxHEVzQcmgZsOkV4LejjOFgRWrcNujXxmsZ9l81O UhmyHpln8KVo0pSg5NhbANxyv8N7X8mA7VF5khIJf9qazavQPdB4tqDVjo1VsRkYPwy6Wj 9Gl11eYfknPDjCq6IoJUpuHvYIZ+470= X-MC-Unique: GsDzzPyvM8GwGBifPta5bA-1 X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=mmPTCZCZTk49wlE2CkFHTKouE1bMT/b1++uEYojsNak=; b=mw9l5ERFkd36h7v7L4Zw70sq5rlTzkkLcltx1mgdejQkqreeDYZPe4n2pk8UjRUqO5 DPTMHXb2dMcEA2HTHMnsgz0XYjWZm7sTeXRYmI6Yts+a4xX8Z/fgVT29PI0NULZ+oVF2 R3YttlAFm/yzXXYLVmAj7PG/k0X1dO9bOjXq+ZOtiEwTB4DqOo6/7cKDihnGn84KxJ5q +iB97MnHUPbwuzOcpfelxAf7DQvMPhlu2Dw0KIREd3CZXGg13LR91J+aS43xprKgF3CD fMt+ICW+kLODoxBQDwy8KD41BP8nA6SfTJTfPo32PMt9krRwWzG7wflJ6C887Xyd6EAm /LRw== X-Gm-Message-State: AOAM5332VJOxnzeaErNr+hr6uQ+0M2kaamhrj4RspaDmREMFfxFnovVg qLKxM/P50vse4YL6tLWTW+bNs0q5LWu38BUFjW7Xt/8RLTBn2nbOwXTL5cuUEvGpzccHEZQHKaA gOJ+7FGa9KC3te4jZUQ/62VuWv/wgxKycpRS0zvRHpz9VQVlxtdGT8vaLhnI41rCA X-Received: by 2002:a05:6a00:1d13:b0:4e1:7958:b59d with SMTP id a19-20020a056a001d1300b004e17958b59dmr25986876pfx.68.1646124013457; Tue, 01 Mar 2022 00:40:13 -0800 (PST) X-Google-Smtp-Source: ABdhPJwBNmb33fvj6d3x2ezOtTmnihBhAqBjq6Lffxsm8aN5WvajnQbMc/fYql+orqFlPVFBqtBzbQ== X-Received: by 2002:a05:6a00:1d13:b0:4e1:7958:b59d with SMTP id a19-20020a056a001d1300b004e17958b59dmr25986851pfx.68.1646124013048; Tue, 01 Mar 2022 00:40:13 -0800 (PST) From: Peter Xu To: qemu-devel@nongnu.org Subject: [PATCH v2 14/25] migration: Add migration_incoming_transport_cleanup() Date: Tue, 1 Mar 2022 16:39:14 +0800 Message-Id: <20220301083925.33483-15-peterx@redhat.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20220301083925.33483-1-peterx@redhat.com> References: <20220301083925.33483-1-peterx@redhat.com> MIME-Version: 1.0 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=peterx@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=peterx@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -28 X-Spam_score: -2.9 X-Spam_bar: -- X-Spam_report: (-2.9 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.082, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H5=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Juan Quintela , "Dr . David Alan Gilbert" , peterx@redhat.com, Leonardo Bras Soares Passos Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZM-MESSAGEID: 1646125008463100001 Content-Type: text/plain; charset="utf-8" Add a helper to cleanup the transport listener. When do it, we should also null-ify the cleanup hook and the data, then it's even safe to call it multiple times. Move the socket_address_list cleanup altogether, because that's a mirror of= the listener channels and only for the purpose of query-migrate. Hence when someone wants to cleanup the listener transport, it should also want to cle= anup the socket list too, always. No functional change intended. Reviewed-by: Dr. David Alan Gilbert Signed-off-by: Peter Xu --- migration/migration.c | 22 ++++++++++++++-------- migration/migration.h | 1 + 2 files changed, 15 insertions(+), 8 deletions(-) diff --git a/migration/migration.c b/migration/migration.c index b2e6446457..6bb321cdd3 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -279,6 +279,19 @@ MigrationIncomingState *migration_incoming_get_current= (void) return current_incoming; } =20 +void migration_incoming_transport_cleanup(MigrationIncomingState *mis) +{ + if (mis->socket_address_list) { + qapi_free_SocketAddressList(mis->socket_address_list); + mis->socket_address_list =3D NULL; + } + + if (mis->transport_cleanup) { + mis->transport_cleanup(mis->transport_data); + mis->transport_data =3D mis->transport_cleanup =3D NULL; + } +} + void migration_incoming_state_destroy(void) { struct MigrationIncomingState *mis =3D migration_incoming_get_current(= ); @@ -299,10 +312,8 @@ void migration_incoming_state_destroy(void) g_array_free(mis->postcopy_remote_fds, TRUE); mis->postcopy_remote_fds =3D NULL; } - if (mis->transport_cleanup) { - mis->transport_cleanup(mis->transport_data); - } =20 + migration_incoming_transport_cleanup(mis); qemu_event_reset(&mis->main_thread_load_event); =20 if (mis->page_requested) { @@ -310,11 +321,6 @@ void migration_incoming_state_destroy(void) mis->page_requested =3D NULL; } =20 - if (mis->socket_address_list) { - qapi_free_SocketAddressList(mis->socket_address_list); - mis->socket_address_list =3D NULL; - } - yank_unregister_instance(MIGRATION_YANK_INSTANCE); } =20 diff --git a/migration/migration.h b/migration/migration.h index d677a750c9..f17ccc657c 100644 --- a/migration/migration.h +++ b/migration/migration.h @@ -166,6 +166,7 @@ struct MigrationIncomingState { =20 MigrationIncomingState *migration_incoming_get_current(void); void migration_incoming_state_destroy(void); +void migration_incoming_transport_cleanup(MigrationIncomingState *mis); /* * Functions to work with blocktime context */ --=20 2.32.0 From nobody Mon Feb 9 00:30:16 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1646125736983860.8208575982968; Tue, 1 Mar 2022 01:08:56 -0800 (PST) Received: from localhost ([::1]:45590 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nOyV6-0005Ol-8y for importer@patchew.org; Tue, 01 Mar 2022 04:08:56 -0500 Received: from eggs.gnu.org ([209.51.188.92]:54714) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nOy3R-0004Fv-S8 for qemu-devel@nongnu.org; Tue, 01 Mar 2022 03:40:21 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]:55743) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nOy3O-0008Fe-W4 for qemu-devel@nongnu.org; Tue, 01 Mar 2022 03:40:20 -0500 Received: from mail-pg1-f198.google.com (mail-pg1-f198.google.com [209.85.215.198]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-505-eygwp5KBMjC8DkbA3q1huw-1; Tue, 01 Mar 2022 03:40:17 -0500 Received: by mail-pg1-f198.google.com with SMTP id v4-20020a63f844000000b003745fd0919aso8148862pgj.20 for ; Tue, 01 Mar 2022 00:40:17 -0800 (PST) Received: from localhost.localdomain ([94.177.118.144]) by smtp.gmail.com with ESMTPSA id l1-20020a17090aec0100b001bc6d8bb27dsm1439987pjy.37.2022.03.01.00.40.13 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Tue, 01 Mar 2022 00:40:15 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1646124018; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=VC80P80l7wFMnNypuT8Iab0jvjyzZ7olmZZPDsWbNdY=; b=UngRo37f1MFbAGvd9z3lSMhb6URxiqsJVKHcxVHEux4IySxHC6s2u2zgDwCPV0LSGRA2IR LXjfLpRYFpdLgGrbQHlmQOO/TJKENKfHJw9Xm3TDtuk9HGGLRvbhJ2+v3uYikcxUNa+el4 e/MgvzZeZR7G/XnIC6heez7RYKc/0/g= X-MC-Unique: eygwp5KBMjC8DkbA3q1huw-1 X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=VC80P80l7wFMnNypuT8Iab0jvjyzZ7olmZZPDsWbNdY=; b=TQA0HqqHIdSId3KwwAkJgaS90aNodDzycYA0wpc73aB4Cx6tB8N1GTXTptsm9jpA0u lcn3DnWlGA1DgzonIwW1ShXkUy+ldad871rUHrleBddm6DZe1UOgjaiTsnrh2FKEjgqs rChTbhV8o2dm9cAQDGzKAGSAZstULprQ0VLnNyvn9qbEMq1mghT97oK/gj6J+m7O0Wkb LkUikQJsj4USlP6DQrzUR6RipAR/G/EBpI7VmWaHSkv016TThbbHuK5HqyU0lwqvZziK uacaNbjAfBKC3ovlFhQbTDHTn10p1HOX97CjM2G3vDSNBzKK8PJrkU4ZK8JtnnD3q1xM kUgQ== X-Gm-Message-State: AOAM531fafjkFmVG0LA9ZxqnzaTKpzKIa2WSuHPqtqtEy27S7zhA66EC B/jaWa9hFo6B9cZm/ofHwEZTUCkWRJ9W5PeBg3FH52kSICTEIlYSK7y5rVWZ/L5ngQcwjYQynPR wqstPCfBiiEuJS6rBPPv/ppx/faW7f96x0ORNjrcowdO6DbwXu8cgolPHcLcZ7qD9 X-Received: by 2002:a17:902:6acb:b0:150:c60:294a with SMTP id i11-20020a1709026acb00b001500c60294amr24633293plt.71.1646124016231; Tue, 01 Mar 2022 00:40:16 -0800 (PST) X-Google-Smtp-Source: ABdhPJxIMDjWYvWQ2AJgJcmyODC4UbipY2XIyQEX3Ffz7aDb4yKxcTkxGlav7W0WFgkZalEHliEuJA== X-Received: by 2002:a17:902:6acb:b0:150:c60:294a with SMTP id i11-20020a1709026acb00b001500c60294amr24633257plt.71.1646124015672; Tue, 01 Mar 2022 00:40:15 -0800 (PST) From: Peter Xu To: qemu-devel@nongnu.org Subject: [PATCH v2 15/25] migration: Allow migrate-recover to run multiple times Date: Tue, 1 Mar 2022 16:39:15 +0800 Message-Id: <20220301083925.33483-16-peterx@redhat.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20220301083925.33483-1-peterx@redhat.com> References: <20220301083925.33483-1-peterx@redhat.com> MIME-Version: 1.0 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=peterx@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=peterx@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -28 X-Spam_score: -2.9 X-Spam_bar: -- X-Spam_report: (-2.9 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.082, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H5=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Juan Quintela , "Dr . David Alan Gilbert" , peterx@redhat.com, Leonardo Bras Soares Passos Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZM-MESSAGEID: 1646125738907100002 Content-Type: text/plain; charset="utf-8" Previously migration didn't have an easy way to cleanup the listening transport, migrate recovery only allows to execute once. That's done with a trick flag in postcopy_recover_triggered. Now the facility is already there. Drop postcopy_recover_triggered and instead allows a new migrate-recover to release the previous listener transport. Reviewed-by: Dr. David Alan Gilbert Signed-off-by: Peter Xu --- migration/migration.c | 13 ++----------- migration/migration.h | 1 - migration/savevm.c | 3 --- 3 files changed, 2 insertions(+), 15 deletions(-) diff --git a/migration/migration.c b/migration/migration.c index 6bb321cdd3..16086897aa 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -2159,11 +2159,8 @@ void qmp_migrate_recover(const char *uri, Error **er= rp) return; } =20 - if (qatomic_cmpxchg(&mis->postcopy_recover_triggered, - false, true) =3D=3D true) { - error_setg(errp, "Migrate recovery is triggered already"); - return; - } + /* If there's an existing transport, release it */ + migration_incoming_transport_cleanup(mis); =20 /* * Note that this call will never start a real migration; it will @@ -2171,12 +2168,6 @@ void qmp_migrate_recover(const char *uri, Error **er= rp) * to continue using that newly established channel. */ qemu_start_incoming_migration(uri, errp); - - /* Safe to dereference with the assert above */ - if (*errp) { - /* Reset the flag so user could still retry */ - qatomic_set(&mis->postcopy_recover_triggered, false); - } } =20 void qmp_migrate_pause(Error **errp) diff --git a/migration/migration.h b/migration/migration.h index f17ccc657c..a863032b71 100644 --- a/migration/migration.h +++ b/migration/migration.h @@ -139,7 +139,6 @@ struct MigrationIncomingState { struct PostcopyBlocktimeContext *blocktime_ctx; =20 /* notify PAUSED postcopy incoming migrations to try to continue */ - bool postcopy_recover_triggered; QemuSemaphore postcopy_pause_sem_dst; QemuSemaphore postcopy_pause_sem_fault; =20 diff --git a/migration/savevm.c b/migration/savevm.c index 967ff80547..254aa78234 100644 --- a/migration/savevm.c +++ b/migration/savevm.c @@ -2589,9 +2589,6 @@ static bool postcopy_pause_incoming(MigrationIncoming= State *mis) =20 assert(migrate_postcopy_ram()); =20 - /* Clear the triggered bit to allow one recovery */ - mis->postcopy_recover_triggered =3D false; - /* * Unregister yank with either from/to src would work, since ioc behin= d it * is the same --=20 2.32.0 From nobody Mon Feb 9 00:30:16 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 164612626705236.103789123934575; Tue, 1 Mar 2022 01:17:47 -0800 (PST) Received: from localhost ([::1]:54470 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nOydd-0003Aq-Gg for importer@patchew.org; Tue, 01 Mar 2022 04:17:45 -0500 Received: from eggs.gnu.org ([209.51.188.92]:54758) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nOy3T-0004Kh-Hm for qemu-devel@nongnu.org; Tue, 01 Mar 2022 03:40:23 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:23544) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nOy3R-0008GO-Jd for qemu-devel@nongnu.org; Tue, 01 Mar 2022 03:40:23 -0500 Received: from mail-pj1-f69.google.com (mail-pj1-f69.google.com [209.85.216.69]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-104-WL3FOHRENfWnZor6p4wMRw-1; Tue, 01 Mar 2022 03:40:20 -0500 Received: by mail-pj1-f69.google.com with SMTP id p5-20020a17090a748500b001bee6752974so140315pjk.8 for ; Tue, 01 Mar 2022 00:40:19 -0800 (PST) Received: from localhost.localdomain ([94.177.118.144]) by smtp.gmail.com with ESMTPSA id l1-20020a17090aec0100b001bc6d8bb27dsm1439987pjy.37.2022.03.01.00.40.16 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Tue, 01 Mar 2022 00:40:17 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1646124020; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=6TAoGPC83WG2WE9uR3fdoGBug6yf6QOx7nZQCRz0NcE=; b=Q+7PIlQ0r6YqsqJf4MJgroWg/FqWJfsDIpE9AgCkTj5aCvnbg0FMzgdVhgAh+WursZR9si qXUEJn8HMnxp8cdN886T+cKDp1X6dwaCPaQjzG1PC+rsfc4x3A8KrjIIvojPT/QUtXApdj tdzMtlCxT21a+EyvQEoqGieCRQ3qIEY= X-MC-Unique: WL3FOHRENfWnZor6p4wMRw-1 X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=6TAoGPC83WG2WE9uR3fdoGBug6yf6QOx7nZQCRz0NcE=; b=TUsNJHHLzwv95CzdNh3zpS0aq0NrlUyu2SXzsPO2nUp+NxEPvuWWI4e/TCVR/P1Vn1 mw6oWNLZyfMXtwCopDIQaXxDiT0rKs71w1vHd5j/3epnCkpQA2OMSF5U57oxJrUldPsB x9P4hch1cAdJWmiZCGRUvj6aMqm4T2FSzDbvV5lRx7sBpkqyUzoOZLAjy8qMc5E3Nzvi wbKPJYn1pg72Z0HFb36gvaenGbnJ7YRYwx/3N7ILfQEo2PqmLArGm3KdCInFApK4CYOj hWb90EeD6mMMGaAjRLf7nFZyaMLLd0iiF3/Dlw2zELK7WORvM+wSyq0rPS8MKigbmEPs TfWA== X-Gm-Message-State: AOAM533g+wwk9oK2RSczqmKcj/CHZdHgrH7OnlR0Y4+q0PtPTuoQraN9 LopqOyUy+ghXGc6M6mx2+6Y8/OIiVpZDtTSvxCZcS9Yac/R0yVw3fYI51Hxy1qaYkpKnL+BABIc pLZQL1zYuuHR48a8DHpZgKLi5bLKy4milCgWjVx2FE914LDKPgAbzlw6eIr2OV9TN X-Received: by 2002:aa7:81c3:0:b0:4f2:6d3f:6158 with SMTP id c3-20020aa781c3000000b004f26d3f6158mr26364367pfn.82.1646124018652; Tue, 01 Mar 2022 00:40:18 -0800 (PST) X-Google-Smtp-Source: ABdhPJwq7FZgTqBlkOhdhz2NkPC8xaZ7IqE+dRb+pqhwfce/6oRmXZXRMCeUvLJbgLNv8VrUpTVSWg== X-Received: by 2002:aa7:81c3:0:b0:4f2:6d3f:6158 with SMTP id c3-20020aa781c3000000b004f26d3f6158mr26364334pfn.82.1646124018224; Tue, 01 Mar 2022 00:40:18 -0800 (PST) From: Peter Xu To: qemu-devel@nongnu.org Subject: [PATCH v2 16/25] migration: Add postcopy-preempt capability Date: Tue, 1 Mar 2022 16:39:16 +0800 Message-Id: <20220301083925.33483-17-peterx@redhat.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20220301083925.33483-1-peterx@redhat.com> References: <20220301083925.33483-1-peterx@redhat.com> MIME-Version: 1.0 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=peterx@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=peterx@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -28 X-Spam_score: -2.9 X-Spam_bar: -- X-Spam_report: (-2.9 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.082, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H5=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Juan Quintela , "Dr . David Alan Gilbert" , peterx@redhat.com, Leonardo Bras Soares Passos Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZM-MESSAGEID: 1646126268571100001 Content-Type: text/plain; charset="utf-8" Firstly, postcopy already preempts precopy due to the fact that we do unqueue_page() first before looking into dirty bits. However that's not enough, e.g., when there're host huge page enabled, when sending a precopy huge page, a postcopy request needs to wait until the who= le huge page that is sending to finish. That could introduce quite some delay, the bigger the huge page is the larger delay it'll bring. This patch adds a new capability to allow postcopy requests to preempt exis= ting precopy page during sending a huge page, so that postcopy requests can be serviced even faster. Meanwhile to send it even faster, bypass the precopy stream by providing a standalone postcopy socket for sending requested pages. Since the new behavior will not be compatible with the old behavior, this w= ill not be the default, it's enabled only when the new capability is set on both src/dst QEMUs. This patch only adds the capability itself, the logic will be added in foll= ow up patches. Reviewed-by: Dr. David Alan Gilbert Signed-off-by: Peter Xu --- migration/migration.c | 23 +++++++++++++++++++++++ migration/migration.h | 1 + qapi/migration.json | 8 +++++++- 3 files changed, 31 insertions(+), 1 deletion(-) diff --git a/migration/migration.c b/migration/migration.c index 16086897aa..4c22bad304 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -1235,6 +1235,11 @@ static bool migrate_caps_check(bool *cap_list, error_setg(errp, "Postcopy is not compatible with ignore-share= d"); return false; } + + if (cap_list[MIGRATION_CAPABILITY_MULTIFD]) { + error_setg(errp, "Multifd is not supported in postcopy"); + return false; + } } =20 if (cap_list[MIGRATION_CAPABILITY_BACKGROUND_SNAPSHOT]) { @@ -1278,6 +1283,13 @@ static bool migrate_caps_check(bool *cap_list, return false; } =20 + if (cap_list[MIGRATION_CAPABILITY_POSTCOPY_PREEMPT]) { + if (!cap_list[MIGRATION_CAPABILITY_POSTCOPY_RAM]) { + error_setg(errp, "Postcopy preempt requires postcopy-ram"); + return false; + } + } + return true; } =20 @@ -2622,6 +2634,15 @@ bool migrate_background_snapshot(void) return s->enabled_capabilities[MIGRATION_CAPABILITY_BACKGROUND_SNAPSHO= T]; } =20 +bool migrate_postcopy_preempt(void) +{ + MigrationState *s; + + s =3D migrate_get_current(); + + return s->enabled_capabilities[MIGRATION_CAPABILITY_POSTCOPY_PREEMPT]; +} + /* migration thread support */ /* * Something bad happened to the RP stream, mark an error @@ -4232,6 +4253,8 @@ static Property migration_properties[] =3D { DEFINE_PROP_MIG_CAP("x-compress", MIGRATION_CAPABILITY_COMPRESS), DEFINE_PROP_MIG_CAP("x-events", MIGRATION_CAPABILITY_EVENTS), DEFINE_PROP_MIG_CAP("x-postcopy-ram", MIGRATION_CAPABILITY_POSTCOPY_RA= M), + DEFINE_PROP_MIG_CAP("x-postcopy-preempt", + MIGRATION_CAPABILITY_POSTCOPY_PREEMPT), DEFINE_PROP_MIG_CAP("x-colo", MIGRATION_CAPABILITY_X_COLO), DEFINE_PROP_MIG_CAP("x-release-ram", MIGRATION_CAPABILITY_RELEASE_RAM), DEFINE_PROP_MIG_CAP("x-block", MIGRATION_CAPABILITY_BLOCK), diff --git a/migration/migration.h b/migration/migration.h index a863032b71..af4bcb19c2 100644 --- a/migration/migration.h +++ b/migration/migration.h @@ -394,6 +394,7 @@ int migrate_decompress_threads(void); bool migrate_use_events(void); bool migrate_postcopy_blocktime(void); bool migrate_background_snapshot(void); +bool migrate_postcopy_preempt(void); =20 /* Sending on the return path - generic and then for each message type */ void migrate_send_rp_shut(MigrationIncomingState *mis, diff --git a/qapi/migration.json b/qapi/migration.json index 5975a0e104..50878b5f3b 100644 --- a/qapi/migration.json +++ b/qapi/migration.json @@ -463,6 +463,12 @@ # procedure starts. The VM RAM is saved with running= VM. # (since 6.0) # +# @postcopy-preempt: If enabled, the migration process will allow postcopy +# requests to preempt precopy stream, so postcopy reque= sts +# will be handled faster. This is a performance featur= e and +# should not affect the correctness of postcopy migrati= on. +# (since 7.0) +# # Features: # @unstable: Members @x-colo and @x-ignore-shared are experimental. # @@ -476,7 +482,7 @@ 'block', 'return-path', 'pause-before-switchover', 'multifd', 'dirty-bitmaps', 'postcopy-blocktime', 'late-block-activate', { 'name': 'x-ignore-shared', 'features': [ 'unstable' ] }, - 'validate-uuid', 'background-snapshot'] } + 'validate-uuid', 'background-snapshot', 'postcopy-preempt'] } =20 ## # @MigrationCapabilityStatus: --=20 2.32.0 From nobody Mon Feb 9 00:30:16 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1646126174294919.3297349750259; Tue, 1 Mar 2022 01:16:14 -0800 (PST) Received: from localhost ([::1]:52268 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nOyc9-0001eL-90 for importer@patchew.org; Tue, 01 Mar 2022 04:16:13 -0500 Received: from eggs.gnu.org ([209.51.188.92]:54788) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nOy3X-0004Oh-SH for qemu-devel@nongnu.org; Tue, 01 Mar 2022 03:40:29 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]:21065) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nOy3V-0008Is-DS for qemu-devel@nongnu.org; Tue, 01 Mar 2022 03:40:27 -0500 Received: from mail-pj1-f69.google.com (mail-pj1-f69.google.com [209.85.216.69]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-321-nPdHNtHbM22OOrZlaCP3GA-1; Tue, 01 Mar 2022 03:40:24 -0500 Received: by mail-pj1-f69.google.com with SMTP id b9-20020a17090aa58900b001b8b14b4aabso1198728pjq.9 for ; Tue, 01 Mar 2022 00:40:23 -0800 (PST) Received: from localhost.localdomain ([94.177.118.144]) by smtp.gmail.com with ESMTPSA id l1-20020a17090aec0100b001bc6d8bb27dsm1439987pjy.37.2022.03.01.00.40.18 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Tue, 01 Mar 2022 00:40:21 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1646124024; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=rfgUcZ6X3iys9RVBrxxjEs8zRK1EZ+AtIIFYh53XXwE=; b=Q5sZaFbXnXrV6PeptM3dEvyE++2kHppUGhRdZ2raXZmYXlVCSNTqS1sI8irFk1a4aZ4rK/ 43bRBTJW1UVps5MZRC654BkLU9YE8EY5EvRVx1uOjrSv3k21dmahjthTrl0PkiuI7KYs2S xnXGv0gcS4YhGis94vOzAJ6uOTI6Viw= X-MC-Unique: nPdHNtHbM22OOrZlaCP3GA-1 X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=rfgUcZ6X3iys9RVBrxxjEs8zRK1EZ+AtIIFYh53XXwE=; b=R5u1/+zGiiQ3Jop+nRKmhHwcHOUTpwZbLugXBimNDARyN8hzG3iMDlx5VQrvd/6ZDq pWOp0qyiBswDEDTsYwuQmaGu34WLwPkWrzef5Eku7kYlc2jltgrYHL87M5/CUug4MQw+ FGmL8/JbrsTy7SddVqOQu0M64ag0UjD0zV7e9RPiviTNwRiWOwhYbVWjxsUw5XSJ+5nn rc4sd9SmMOB8kq6XGlaFqjLm+6jpG1xCcTMmw0cJJVqHyKdSi7ExGYCQ3hiz8t07gSgX CsRe7KQKKRapZ993LA0NqXcm0RaHeX8khIM3VCgctbNrD35IhaKKBcCMQhzdBNjMHgbK q+uA== X-Gm-Message-State: AOAM531a6lp5XeANFJ0mAk7lGIMhLd7E/qDU562I26kbz7Z4xIOoMMmZ 8eZzvw80r3a4FRc8xJ8KCxCBW5g9UI8V8nqNIeETaiujA7L9KlO5QRTR6lrrUieFOAAIzfbQnEl t5k0pdyu15aRNcnTgzgDj0Mqh6TwzTn/RVb4zaz+ULqqPV1vOWdiIZie8xs2tG7au X-Received: by 2002:aa7:8643:0:b0:4d1:d068:ef15 with SMTP id a3-20020aa78643000000b004d1d068ef15mr26337817pfo.9.1646124022438; Tue, 01 Mar 2022 00:40:22 -0800 (PST) X-Google-Smtp-Source: ABdhPJwHifXQDXE6if/zMyhT0BQkGQoXVw8HMOyZpWyppgCGoPQqVNkNgAnV65+MlDowOD1jJGin7w== X-Received: by 2002:aa7:8643:0:b0:4d1:d068:ef15 with SMTP id a3-20020aa78643000000b004d1d068ef15mr26337772pfo.9.1646124021833; Tue, 01 Mar 2022 00:40:21 -0800 (PST) From: Peter Xu To: qemu-devel@nongnu.org Subject: [PATCH v2 17/25] migration: Postcopy preemption preparation on channel creation Date: Tue, 1 Mar 2022 16:39:17 +0800 Message-Id: <20220301083925.33483-18-peterx@redhat.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20220301083925.33483-1-peterx@redhat.com> References: <20220301083925.33483-1-peterx@redhat.com> MIME-Version: 1.0 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=peterx@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=peterx@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -22 X-Spam_score: -2.3 X-Spam_bar: -- X-Spam_report: (-2.3 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.082, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H5=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01, URG_BIZ=0.573 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Juan Quintela , "Dr . David Alan Gilbert" , peterx@redhat.com, Leonardo Bras Soares Passos Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZM-MESSAGEID: 1646126176434100001 Content-Type: text/plain; charset="utf-8" Create a new socket for postcopy to be prepared to send postcopy requested pages via this specific channel, so as to not get blocked by precopy pages. A new thread is also created on dest qemu to receive data from this new cha= nnel based on the ram_load_postcopy() routine. The ram_load_postcopy(POSTCOPY) branch and the thread has not started to function, and that'll be done in follow up patches. Cleanup the new sockets on both src/dst QEMUs, meanwhile look after the new thread too to make sure it'll be recycled properly. Signed-off-by: Peter Xu --- migration/migration.c | 62 +++++++++++++++++++++++---- migration/migration.h | 8 ++++ migration/postcopy-ram.c | 92 ++++++++++++++++++++++++++++++++++++++-- migration/postcopy-ram.h | 10 +++++ migration/ram.c | 25 ++++++++--- migration/ram.h | 4 +- migration/socket.c | 22 +++++++++- migration/socket.h | 1 + migration/trace-events | 3 ++ 9 files changed, 207 insertions(+), 20 deletions(-) diff --git a/migration/migration.c b/migration/migration.c index 4c22bad304..3d7f897b72 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -321,6 +321,12 @@ void migration_incoming_state_destroy(void) mis->page_requested =3D NULL; } =20 + if (mis->postcopy_qemufile_dst) { + migration_ioc_unregister_yank_from_file(mis->postcopy_qemufile_dst= ); + qemu_fclose(mis->postcopy_qemufile_dst); + mis->postcopy_qemufile_dst =3D NULL; + } + yank_unregister_instance(MIGRATION_YANK_INSTANCE); } =20 @@ -714,15 +720,21 @@ void migration_fd_process_incoming(QEMUFile *f, Error= **errp) migration_incoming_process(); } =20 +static bool migration_needs_multiple_sockets(void) +{ + return migrate_use_multifd() || migrate_postcopy_preempt(); +} + void migration_ioc_process_incoming(QIOChannel *ioc, Error **errp) { MigrationIncomingState *mis =3D migration_incoming_get_current(); Error *local_err =3D NULL; bool start_migration; + QEMUFile *f; =20 if (!mis->from_src_file) { /* The first connection (multifd may have multiple) */ - QEMUFile *f =3D qemu_fopen_channel_input(ioc); + f =3D qemu_fopen_channel_input(ioc); =20 if (!migration_incoming_setup(f, errp)) { return; @@ -730,13 +742,18 @@ void migration_ioc_process_incoming(QIOChannel *ioc, = Error **errp) =20 /* * Common migration only needs one channel, so we can start - * right now. Multifd needs more than one channel, we wait. + * right now. Some features need more than one channel, we wait. */ - start_migration =3D !migrate_use_multifd(); + start_migration =3D !migration_needs_multiple_sockets(); } else { /* Multiple connections */ - assert(migrate_use_multifd()); - start_migration =3D multifd_recv_new_channel(ioc, &local_err); + assert(migration_needs_multiple_sockets()); + if (migrate_use_multifd()) { + start_migration =3D multifd_recv_new_channel(ioc, &local_err); + } else if (migrate_postcopy_preempt()) { + f =3D qemu_fopen_channel_input(ioc); + start_migration =3D postcopy_preempt_new_channel(mis, f); + } if (local_err) { error_propagate(errp, local_err); return; @@ -761,11 +778,20 @@ void migration_ioc_process_incoming(QIOChannel *ioc, = Error **errp) bool migration_has_all_channels(void) { MigrationIncomingState *mis =3D migration_incoming_get_current(); - bool all_channels; =20 - all_channels =3D multifd_recv_all_channels_created(); + if (!mis->from_src_file) { + return false; + } + + if (migrate_use_multifd()) { + return multifd_recv_all_channels_created(); + } + + if (migrate_postcopy_preempt()) { + return mis->postcopy_qemufile_dst !=3D NULL; + } =20 - return all_channels && mis->from_src_file !=3D NULL; + return true; } =20 /* @@ -1858,6 +1884,12 @@ static void migrate_fd_cleanup(MigrationState *s) qemu_fclose(tmp); } =20 + if (s->postcopy_qemufile_src) { + migration_ioc_unregister_yank_from_file(s->postcopy_qemufile_src); + qemu_fclose(s->postcopy_qemufile_src); + s->postcopy_qemufile_src =3D NULL; + } + assert(!migration_is_active(s)); =20 if (s->state =3D=3D MIGRATION_STATUS_CANCELLING) { @@ -3233,6 +3265,11 @@ static void migration_completion(MigrationState *s) qemu_savevm_state_complete_postcopy(s->to_dst_file); qemu_mutex_unlock_iothread(); =20 + /* Shutdown the postcopy fast path thread */ + if (migrate_postcopy_preempt()) { + postcopy_preempt_shutdown_file(s); + } + trace_migration_completion_postcopy_end_after_complete(); } else { goto fail; @@ -4120,6 +4157,15 @@ void migrate_fd_connect(MigrationState *s, Error *er= ror_in) } } =20 + /* This needs to be done before resuming a postcopy */ + if (postcopy_preempt_setup(s, &local_err)) { + error_report_err(local_err); + migrate_set_state(&s->state, MIGRATION_STATUS_SETUP, + MIGRATION_STATUS_FAILED); + migrate_fd_cleanup(s); + return; + } + if (resume) { /* Wakeup the main migration thread to do the recovery */ migrate_set_state(&s->state, MIGRATION_STATUS_POSTCOPY_PAUSED, diff --git a/migration/migration.h b/migration/migration.h index af4bcb19c2..caa910d956 100644 --- a/migration/migration.h +++ b/migration/migration.h @@ -23,6 +23,7 @@ #include "io/channel-buffer.h" #include "net/announce.h" #include "qom/object.h" +#include "postcopy-ram.h" =20 struct PostcopyBlocktimeContext; =20 @@ -112,6 +113,11 @@ struct MigrationIncomingState { * enabled. */ unsigned int postcopy_channels; + /* QEMUFile for postcopy only; it'll be handled by a separate thread */ + QEMUFile *postcopy_qemufile_dst; + /* Postcopy priority thread is used to receive postcopy requested page= s */ + QemuThread postcopy_prio_thread; + bool postcopy_prio_thread_created; /* * An array of temp host huge pages to be used, one for each postcopy * channel. @@ -192,6 +198,8 @@ struct MigrationState { QEMUBH *cleanup_bh; /* Protected by qemu_file_lock */ QEMUFile *to_dst_file; + /* Postcopy specific transfer channel */ + QEMUFile *postcopy_qemufile_src; QIOChannelBuffer *bioc; /* * Protects to_dst_file/from_dst_file pointers. We need to make sure = we diff --git a/migration/postcopy-ram.c b/migration/postcopy-ram.c index 32c52f4b1d..df0c02f729 100644 --- a/migration/postcopy-ram.c +++ b/migration/postcopy-ram.c @@ -33,6 +33,9 @@ #include "trace.h" #include "hw/boards.h" #include "exec/ramblock.h" +#include "socket.h" +#include "qemu-file-channel.h" +#include "yank_functions.h" =20 /* Arbitrary limit on size of each discard command, * keeps them around ~200 bytes @@ -567,6 +570,11 @@ int postcopy_ram_incoming_cleanup(MigrationIncomingSta= te *mis) { trace_postcopy_ram_incoming_cleanup_entry(); =20 + if (mis->postcopy_prio_thread_created) { + qemu_thread_join(&mis->postcopy_prio_thread); + mis->postcopy_prio_thread_created =3D false; + } + if (mis->have_fault_thread) { Error *local_err =3D NULL; =20 @@ -1102,8 +1110,13 @@ static int postcopy_temp_pages_setup(MigrationIncomi= ngState *mis) int err, i, channels; void *temp_page; =20 - /* TODO: will be boosted when enable postcopy preemption */ - mis->postcopy_channels =3D 1; + if (migrate_postcopy_preempt()) { + /* If preemption enabled, need extra channel for urgent requests */ + mis->postcopy_channels =3D RAM_CHANNEL_MAX; + } else { + /* Both precopy/postcopy on the same channel */ + mis->postcopy_channels =3D 1; + } =20 channels =3D mis->postcopy_channels; mis->postcopy_tmp_pages =3D g_malloc0_n(sizeof(PostcopyTmpPage), chann= els); @@ -1170,7 +1183,7 @@ int postcopy_ram_incoming_setup(MigrationIncomingStat= e *mis) return -1; } =20 - postcopy_thread_create(mis, &mis->fault_thread, "postcopy/fault", + postcopy_thread_create(mis, &mis->fault_thread, "fault-default", postcopy_ram_fault_thread, QEMU_THREAD_JOINABLE= ); mis->have_fault_thread =3D true; =20 @@ -1185,6 +1198,16 @@ int postcopy_ram_incoming_setup(MigrationIncomingSta= te *mis) return -1; } =20 + if (migrate_postcopy_preempt()) { + /* + * This thread needs to be created after the temp pages because it= 'll fetch + * RAM_CHANNEL_POSTCOPY PostcopyTmpPage immediately. + */ + postcopy_thread_create(mis, &mis->postcopy_prio_thread, "fault-fas= t", + postcopy_preempt_thread, QEMU_THREAD_JOINAB= LE); + mis->postcopy_prio_thread_created =3D true; + } + trace_postcopy_ram_enable_notify(); =20 return 0; @@ -1514,3 +1537,66 @@ void postcopy_unregister_shared_ufd(struct PostCopyF= D *pcfd) } } } + +bool postcopy_preempt_new_channel(MigrationIncomingState *mis, QEMUFile *f= ile) +{ + /* + * The new loading channel has its own threads, so it needs to be + * blocked too. It's by default true, just be explicit. + */ + qemu_file_set_blocking(file, true); + mis->postcopy_qemufile_dst =3D file; + trace_postcopy_preempt_new_channel(); + + /* Start the migration immediately */ + return true; +} + +int postcopy_preempt_setup(MigrationState *s, Error **errp) +{ + QIOChannel *ioc; + + if (!migrate_postcopy_preempt()) { + return 0; + } + + if (!migrate_multi_channels_is_allowed()) { + error_setg(errp, "Postcopy preempt is not supported as current " + "migration stream does not support multi-channels."); + return -1; + } + + ioc =3D socket_send_channel_create_sync(errp); + + if (ioc =3D=3D NULL) { + return -1; + } + + migration_ioc_register_yank(ioc); + s->postcopy_qemufile_src =3D qemu_fopen_channel_output(ioc); + + trace_postcopy_preempt_new_channel(); + + return 0; +} + +void *postcopy_preempt_thread(void *opaque) +{ + MigrationIncomingState *mis =3D opaque; + int ret; + + trace_postcopy_preempt_thread_entry(); + + rcu_register_thread(); + + qemu_sem_post(&mis->thread_sync_sem); + + /* Sending RAM_SAVE_FLAG_EOS to terminate this thread */ + ret =3D ram_load_postcopy(mis->postcopy_qemufile_dst, RAM_CHANNEL_POST= COPY); + + rcu_unregister_thread(); + + trace_postcopy_preempt_thread_exit(); + + return ret =3D=3D 0 ? NULL : (void *)-1; +} diff --git a/migration/postcopy-ram.h b/migration/postcopy-ram.h index 07684c0e1d..34b1080cde 100644 --- a/migration/postcopy-ram.h +++ b/migration/postcopy-ram.h @@ -183,4 +183,14 @@ int postcopy_wake_shared(struct PostCopyFD *pcfd, uint= 64_t client_addr, int postcopy_request_shared_page(struct PostCopyFD *pcfd, RAMBlock *rb, uint64_t client_addr, uint64_t offset); =20 +/* Hard-code channels for now for postcopy preemption */ +enum PostcopyChannels { + RAM_CHANNEL_PRECOPY =3D 0, + RAM_CHANNEL_POSTCOPY =3D 1, + RAM_CHANNEL_MAX, +}; + +bool postcopy_preempt_new_channel(MigrationIncomingState *mis, QEMUFile *f= ile); +int postcopy_preempt_setup(MigrationState *s, Error **errp); + #endif diff --git a/migration/ram.c b/migration/ram.c index 5cb5dfc2cc..713ef6e421 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -3644,15 +3644,15 @@ int ram_postcopy_incoming_init(MigrationIncomingSta= te *mis) * rcu_read_lock is taken prior to this being called. * * @f: QEMUFile where to send the data + * @channel: the channel to use for loading */ -int ram_load_postcopy(QEMUFile *f) +int ram_load_postcopy(QEMUFile *f, int channel) { int flags =3D 0, ret =3D 0; bool place_needed =3D false; bool matches_target_page_size =3D false; MigrationIncomingState *mis =3D migration_incoming_get_current(); - /* Currently we only use channel 0. TODO: use all the channels */ - PostcopyTmpPage *tmp_page =3D &mis->postcopy_tmp_pages[0]; + PostcopyTmpPage *tmp_page =3D &mis->postcopy_tmp_pages[channel]; =20 while (!ret && !(flags & RAM_SAVE_FLAG_EOS)) { ram_addr_t addr; @@ -3717,10 +3717,10 @@ int ram_load_postcopy(QEMUFile *f) } else if (tmp_page->host_addr !=3D host_page_from_ram_block_offset(block, addr)) { /* not the 1st TP within the HP */ - error_report("Non-same host page detected. Target host pa= ge %p, " - "received host page %p " + error_report("Non-same host page detected on channel %d: " + "Target host page %p, received host page %p " "(rb %s offset 0x"RAM_ADDR_FMT" target_pages = %d)", - tmp_page->host_addr, + channel, tmp_page->host_addr, host_page_from_ram_block_offset(block, addr), block->idstr, addr, tmp_page->target_pages); ret =3D -EINVAL; @@ -4107,7 +4107,12 @@ static int ram_load(QEMUFile *f, void *opaque, int v= ersion_id) */ WITH_RCU_READ_LOCK_GUARD() { if (postcopy_running) { - ret =3D ram_load_postcopy(f); + /* + * Note! Here RAM_CHANNEL_PRECOPY is the precopy channel of + * postcopy migration, we have another RAM_CHANNEL_POSTCOPY to + * service fast page faults. + */ + ret =3D ram_load_postcopy(f, RAM_CHANNEL_PRECOPY); } else { ret =3D ram_load_precopy(f); } @@ -4269,6 +4274,12 @@ static int ram_resume_prepare(MigrationState *s, voi= d *opaque) return 0; } =20 +void postcopy_preempt_shutdown_file(MigrationState *s) +{ + qemu_put_be64(s->postcopy_qemufile_src, RAM_SAVE_FLAG_EOS); + qemu_fflush(s->postcopy_qemufile_src); +} + static SaveVMHandlers savevm_ram_handlers =3D { .save_setup =3D ram_save_setup, .save_live_iterate =3D ram_save_iterate, diff --git a/migration/ram.h b/migration/ram.h index ded0a3a086..5d90945a6e 100644 --- a/migration/ram.h +++ b/migration/ram.h @@ -61,7 +61,7 @@ void ram_postcopy_send_discard_bitmap(MigrationState *ms); /* For incoming postcopy discard */ int ram_discard_range(const char *block_name, uint64_t start, size_t lengt= h); int ram_postcopy_incoming_init(MigrationIncomingState *mis); -int ram_load_postcopy(QEMUFile *f); +int ram_load_postcopy(QEMUFile *f, int channel); =20 void ram_handle_compressed(void *host, uint8_t ch, uint64_t size); =20 @@ -73,6 +73,8 @@ int64_t ramblock_recv_bitmap_send(QEMUFile *file, const char *block_name); int ram_dirty_bitmap_reload(MigrationState *s, RAMBlock *rb); bool ramblock_page_is_discarded(RAMBlock *rb, ram_addr_t start); +void postcopy_preempt_shutdown_file(MigrationState *s); +void *postcopy_preempt_thread(void *opaque); =20 /* ram cache */ int colo_init_ram_cache(void); diff --git a/migration/socket.c b/migration/socket.c index 05705a32d8..a7f345b353 100644 --- a/migration/socket.c +++ b/migration/socket.c @@ -26,7 +26,7 @@ #include "io/channel-socket.h" #include "io/net-listener.h" #include "trace.h" - +#include "postcopy-ram.h" =20 struct SocketOutgoingArgs { SocketAddress *saddr; @@ -39,6 +39,24 @@ void socket_send_channel_create(QIOTaskFunc f, void *dat= a) f, data, NULL, NULL); } =20 +QIOChannel *socket_send_channel_create_sync(Error **errp) +{ + QIOChannelSocket *sioc =3D qio_channel_socket_new(); + + if (!outgoing_args.saddr) { + object_unref(OBJECT(sioc)); + error_setg(errp, "Initial sock address not set!"); + return NULL; + } + + if (qio_channel_socket_connect_sync(sioc, outgoing_args.saddr, errp) <= 0) { + object_unref(OBJECT(sioc)); + return NULL; + } + + return QIO_CHANNEL(sioc); +} + int socket_send_channel_destroy(QIOChannel *send) { /* Remove channel */ @@ -158,6 +176,8 @@ socket_start_incoming_migration_internal(SocketAddress = *saddr, =20 if (migrate_use_multifd()) { num =3D migrate_multifd_channels(); + } else if (migrate_postcopy_preempt()) { + num =3D RAM_CHANNEL_MAX; } =20 if (qio_net_listener_open_sync(listener, saddr, num, errp) < 0) { diff --git a/migration/socket.h b/migration/socket.h index 891dbccceb..dc54df4e6c 100644 --- a/migration/socket.h +++ b/migration/socket.h @@ -21,6 +21,7 @@ #include "io/task.h" =20 void socket_send_channel_create(QIOTaskFunc f, void *data); +QIOChannel *socket_send_channel_create_sync(Error **errp); int socket_send_channel_destroy(QIOChannel *send); =20 void socket_start_incoming_migration(const char *str, Error **errp); diff --git a/migration/trace-events b/migration/trace-events index 1aec580e92..4474a76614 100644 --- a/migration/trace-events +++ b/migration/trace-events @@ -278,6 +278,9 @@ postcopy_request_shared_page(const char *sharer, const = char *rb, uint64_t rb_off postcopy_request_shared_page_present(const char *sharer, const char *rb, u= int64_t rb_offset) "%s already %s offset 0x%"PRIx64 postcopy_wake_shared(uint64_t client_addr, const char *rb) "at 0x%"PRIx64"= in %s" postcopy_page_req_del(void *addr, int count) "resolved page req %p total %= d" +postcopy_preempt_new_channel(void) "" +postcopy_preempt_thread_entry(void) "" +postcopy_preempt_thread_exit(void) "" =20 get_mem_fault_cpu_index(int cpu, uint32_t pid) "cpu: %d, pid: %u" =20 --=20 2.32.0 From nobody Mon Feb 9 00:30:16 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1646126536658919.0620970613469; Tue, 1 Mar 2022 01:22:16 -0800 (PST) Received: from localhost ([::1]:34874 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nOyhz-0000t4-KO for importer@patchew.org; Tue, 01 Mar 2022 04:22:15 -0500 Received: from eggs.gnu.org ([209.51.188.92]:54812) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nOy3a-0004Pu-P9 for qemu-devel@nongnu.org; Tue, 01 Mar 2022 03:40:30 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:42522) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nOy3Y-0008KH-BB for qemu-devel@nongnu.org; Tue, 01 Mar 2022 03:40:30 -0500 Received: from mail-pl1-f197.google.com (mail-pl1-f197.google.com [209.85.214.197]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-356-QvzOu26-PqmwS0on_R9rkw-1; Tue, 01 Mar 2022 03:40:26 -0500 Received: by mail-pl1-f197.google.com with SMTP id b5-20020a170902e94500b0014f6d0a417bso5933917pll.6 for ; Tue, 01 Mar 2022 00:40:26 -0800 (PST) Received: from localhost.localdomain ([94.177.118.144]) by smtp.gmail.com with ESMTPSA id l1-20020a17090aec0100b001bc6d8bb27dsm1439987pjy.37.2022.03.01.00.40.22 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Tue, 01 Mar 2022 00:40:24 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1646124027; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=E62fNsyk6DLRcGcNcW2TJv7h6f9bfIuEPKahUmmkOG0=; b=BZURNWkJbY5W899Yg3ZrvNsgars/jNku4o5aOYqT1nk0g7r6+jYKTzgOokYRvGrNkRV71Z vYRVkr3hkQHbhOhcwQdm2E1Og5smiJFOYXSo2a7D6oySaSVCyRcWOvvbTJfq1uAZb6qHca gEfHmGvPcU3NcaCzRP/HN8o4piC8IVs= X-MC-Unique: QvzOu26-PqmwS0on_R9rkw-1 X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=E62fNsyk6DLRcGcNcW2TJv7h6f9bfIuEPKahUmmkOG0=; b=HS1ySmBQ/l9ZbrYT4LMV63IaB3nGyN1fwISkM/xlLq7eFadfyUcbrnDVfiLxCNzLkb SeLIk/pS6dWRlengXxwdaMWkNBYQZNUZEnX94ki7yD2RTdVBDkYwO7ahz3YKLE/m5/DI sOalwPgFMNlyNIfBBfJ5lQwR+XVucgnv+TmO9BlivKXQMT9AFu75+Q+hG/cjBftsdQuT 8VMJO5uQO8ZsGIua1jdZlLYd2E9gYzuz/EITd1uLR4Pv9LvPzwBDrs71nyBoN1Zv1NIK Y9T1vsZs2adN3zoEny6CCLxc/3BpdBtMA16rLmC8FqvGH8tV09cxmYRJAHbK6sDFtqyI T5/Q== X-Gm-Message-State: AOAM5319iuEkUm5uiXgxVD+CQCC5Ydn0lgiw+5WU5ZW8r9cPsZIsVwo+ R+IW0btAHEovwCjnfuGtMOdSY3Vdbo/OElBARidzyjDBcw5nyK/T1bidLmraQ8Z95L/kSvS1pD9 yDvGfbO6D62LPLDqxpkaD7AcVMv2dDUT4ffSrDkK9sNTD2DQ1mdDeMKKJUrJGioAW X-Received: by 2002:a17:902:cf02:b0:14f:e0c2:1514 with SMTP id i2-20020a170902cf0200b0014fe0c21514mr24665574plg.90.1646124025164; Tue, 01 Mar 2022 00:40:25 -0800 (PST) X-Google-Smtp-Source: ABdhPJxQhjOlDhlYeVXkODHmEMb5ov8MD17gqw+rt5+h4G25YDQ/TJAA8HQbRuNZj0LV0r6AJaf1gA== X-Received: by 2002:a17:902:cf02:b0:14f:e0c2:1514 with SMTP id i2-20020a170902cf0200b0014fe0c21514mr24665526plg.90.1646124024416; Tue, 01 Mar 2022 00:40:24 -0800 (PST) From: Peter Xu To: qemu-devel@nongnu.org Subject: [PATCH v2 18/25] migration: Postcopy preemption enablement Date: Tue, 1 Mar 2022 16:39:18 +0800 Message-Id: <20220301083925.33483-19-peterx@redhat.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20220301083925.33483-1-peterx@redhat.com> References: <20220301083925.33483-1-peterx@redhat.com> MIME-Version: 1.0 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=peterx@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=peterx@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -28 X-Spam_score: -2.9 X-Spam_bar: -- X-Spam_report: (-2.9 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.082, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H5=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Juan Quintela , "Dr . David Alan Gilbert" , peterx@redhat.com, Leonardo Bras Soares Passos Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZM-MESSAGEID: 1646126538736100001 Content-Type: text/plain; charset="utf-8" This patch enables postcopy-preempt feature. It contains two major changes to the migration logic: (1) Postcopy requests are now sent via a different socket from precopy background migration stream, so as to be isolated from very high page request delays. (2) For huge page enabled hosts: when there's postcopy requests, they can n= ow intercept a partial sending of huge host pages on src QEMU. After this patch, we'll live migrate a VM with two channels for postcopy: (= 1) PRECOPY channel, which is the default channel that transfers background pag= es; and (2) POSTCOPY channel, which only transfers requested pages. There's no strict rule of which channel to use, e.g., if a requested page is already being transferred on precopy channel, then we will keep using the s= ame precopy channel to transfer the page even if it's explicitly requested. In= 99% of the cases we'll prioritize the channels so we send requested page via the postcopy channel as long as possible. On the source QEMU, when we found a postcopy request, we'll interrupt the PRECOPY channel sending process and quickly switch to the POSTCOPY channel. After we serviced all the high priority postcopy pages, we'll switch back to PRECOPY channel so that we'll continue to send the interrupted huge page ag= ain. There's no new thread introduced on src QEMU. On the destination QEMU, one new thread is introduced to receive page data = from the postcopy specific socket (done in the preparation patch). This patch has a side effect: after sending postcopy pages, previously we'll assume the guest will access follow up pages so we'll keep sending from the= re. Now it's changed. Instead of going on with a postcopy requested page, we'l= l go back and continue sending the precopy huge page (which can be intercepted b= y a postcopy request so the huge page can be sent partially before). Whether that's a problem is debatable, because "assuming the guest will continue to access the next page" may not really suite when huge pages are used, especially if the huge page is large (e.g. 1GB pages). So that local= ity hint is much meaningless if huge pages are used. Reviewed-by: Dr. David Alan Gilbert Signed-off-by: Peter Xu --- migration/migration.c | 2 + migration/migration.h | 2 +- migration/ram.c | 250 +++++++++++++++++++++++++++++++++++++++-- migration/trace-events | 7 ++ 4 files changed, 252 insertions(+), 9 deletions(-) diff --git a/migration/migration.c b/migration/migration.c index 3d7f897b72..d20db04097 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -3153,6 +3153,8 @@ static int postcopy_start(MigrationState *ms) MIGRATION_STATUS_FAILED); } =20 + trace_postcopy_preempt_enabled(migrate_postcopy_preempt()); + return ret; =20 fail_closefb: diff --git a/migration/migration.h b/migration/migration.h index caa910d956..b8aacfe3af 100644 --- a/migration/migration.h +++ b/migration/migration.h @@ -68,7 +68,7 @@ typedef struct { struct MigrationIncomingState { QEMUFile *from_src_file; /* Previously received RAM's RAMBlock pointer */ - RAMBlock *last_recv_block; + RAMBlock *last_recv_block[RAM_CHANNEL_MAX]; /* A hook to allow cleanup at the end of incoming migration */ void *transport_data; void (*transport_cleanup)(void *data); diff --git a/migration/ram.c b/migration/ram.c index 713ef6e421..53dfd9be38 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -295,6 +295,20 @@ struct RAMSrcPageRequest { QSIMPLEQ_ENTRY(RAMSrcPageRequest) next_req; }; =20 +typedef struct { + /* + * Cached ramblock/offset values if preempted. They're only meaningfu= l if + * preempted=3D=3Dtrue below. + */ + RAMBlock *ram_block; + unsigned long ram_page; + /* + * Whether a postcopy preemption just happened. Will be reset after + * precopy recovered to background migration. + */ + bool preempted; +} PostcopyPreemptState; + /* State of RAM for migration */ struct RAMState { /* QEMUFile used for this migration */ @@ -349,6 +363,14 @@ struct RAMState { /* Queue of outstanding page requests from the destination */ QemuMutex src_page_req_mutex; QSIMPLEQ_HEAD(, RAMSrcPageRequest) src_page_requests; + + /* Postcopy preemption informations */ + PostcopyPreemptState postcopy_preempt_state; + /* + * Current channel we're using on src VM. Only valid if postcopy-pree= mpt + * is enabled. + */ + unsigned int postcopy_channel; }; typedef struct RAMState RAMState; =20 @@ -356,6 +378,11 @@ static RAMState *ram_state; =20 static NotifierWithReturnList precopy_notifier_list; =20 +static void postcopy_preempt_reset(RAMState *rs) +{ + memset(&rs->postcopy_preempt_state, 0, sizeof(PostcopyPreemptState)); +} + /* Whether postcopy has queued requests? */ static bool postcopy_has_request(RAMState *rs) { @@ -1947,6 +1974,55 @@ void ram_write_tracking_stop(void) } #endif /* defined(__linux__) */ =20 +/* + * Check whether two addr/offset of the ramblock falls onto the same host = huge + * page. Returns true if so, false otherwise. + */ +static bool offset_on_same_huge_page(RAMBlock *rb, uint64_t addr1, + uint64_t addr2) +{ + size_t page_size =3D qemu_ram_pagesize(rb); + + addr1 =3D ROUND_DOWN(addr1, page_size); + addr2 =3D ROUND_DOWN(addr2, page_size); + + return addr1 =3D=3D addr2; +} + +/* + * Whether a previous preempted precopy huge page contains current request= ed + * page? Returns true if so, false otherwise. + * + * This should really happen very rarely, because it means when we were se= nding + * during background migration for postcopy we're sending exactly the page= that + * some vcpu got faulted on on dest node. When it happens, we probably do= n't + * need to do much but drop the request, because we know right after we re= store + * the precopy stream it'll be serviced. It'll slightly affect the order = of + * postcopy requests to be serviced (e.g. it'll be the same as we move cur= rent + * request to the end of the queue) but it shouldn't be a big deal. The m= ost + * imporant thing is we can _never_ try to send a partial-sent huge page o= n the + * POSTCOPY channel again, otherwise that huge page will got "split brain"= on + * two channels (PRECOPY, POSTCOPY). + */ +static bool postcopy_preempted_contains(RAMState *rs, RAMBlock *block, + ram_addr_t offset) +{ + PostcopyPreemptState *state =3D &rs->postcopy_preempt_state; + + /* No preemption at all? */ + if (!state->preempted) { + return false; + } + + /* Not even the same ramblock? */ + if (state->ram_block !=3D block) { + return false; + } + + return offset_on_same_huge_page(block, offset, + state->ram_page << TARGET_PAGE_BITS); +} + /** * get_queued_page: unqueue a page from the postcopy requests * @@ -1962,9 +2038,17 @@ static bool get_queued_page(RAMState *rs, PageSearch= Status *pss) RAMBlock *block; ram_addr_t offset; =20 +again: block =3D unqueue_page(rs, &offset); =20 - if (!block) { + if (block) { + /* See comment above postcopy_preempted_contains() */ + if (postcopy_preempted_contains(rs, block, offset)) { + trace_postcopy_preempt_hit(block->idstr, offset); + /* This request is dropped */ + goto again; + } + } else { /* * Poll write faults too if background snapshot is enabled; that's * when we have vcpus got blocked by the write protected pages. @@ -2180,6 +2264,117 @@ static int ram_save_target_page(RAMState *rs, PageS= earchStatus *pss) return ram_save_page(rs, pss); } =20 +static bool postcopy_needs_preempt(RAMState *rs, PageSearchStatus *pss) +{ + /* Not enabled eager preempt? Then never do that. */ + if (!migrate_postcopy_preempt()) { + return false; + } + + /* If the ramblock we're sending is a small page? Never bother. */ + if (qemu_ram_pagesize(pss->block) =3D=3D TARGET_PAGE_SIZE) { + return false; + } + + /* Not in postcopy at all? */ + if (!migration_in_postcopy()) { + return false; + } + + /* + * If we're already handling a postcopy request, don't preempt as this= page + * has got the same high priority. + */ + if (pss->postcopy_requested) { + return false; + } + + /* If there's postcopy requests, then check it up! */ + return postcopy_has_request(rs); +} + +/* Returns true if we preempted precopy, false otherwise */ +static void postcopy_do_preempt(RAMState *rs, PageSearchStatus *pss) +{ + PostcopyPreemptState *p_state =3D &rs->postcopy_preempt_state; + + trace_postcopy_preempt_triggered(pss->block->idstr, pss->page); + + /* + * Time to preempt precopy. Cache current PSS into preempt state, so t= hat + * after handling the postcopy pages we can recover to it. We need to= do + * so because the dest VM will have partial of the precopy huge page k= ept + * over in its tmp huge page caches; better move on with it when we ca= n. + */ + p_state->ram_block =3D pss->block; + p_state->ram_page =3D pss->page; + p_state->preempted =3D true; +} + +/* Whether we're preempted by a postcopy request during sending a huge pag= e */ +static bool postcopy_preempt_triggered(RAMState *rs) +{ + return rs->postcopy_preempt_state.preempted; +} + +static void postcopy_preempt_restore(RAMState *rs, PageSearchStatus *pss) +{ + PostcopyPreemptState *state =3D &rs->postcopy_preempt_state; + + assert(state->preempted); + + pss->block =3D state->ram_block; + pss->page =3D state->ram_page; + /* This is not a postcopy request but restoring previous precopy */ + pss->postcopy_requested =3D false; + + trace_postcopy_preempt_restored(pss->block->idstr, pss->page); + + /* Reset preempt state, most importantly, set preempted=3D=3Dfalse */ + postcopy_preempt_reset(rs); +} + +static void postcopy_preempt_choose_channel(RAMState *rs, PageSearchStatus= *pss) +{ + MigrationState *s =3D migrate_get_current(); + unsigned int channel; + QEMUFile *next; + + channel =3D pss->postcopy_requested ? + RAM_CHANNEL_POSTCOPY : RAM_CHANNEL_PRECOPY; + + if (channel !=3D rs->postcopy_channel) { + if (channel =3D=3D RAM_CHANNEL_PRECOPY) { + next =3D s->to_dst_file; + } else { + next =3D s->postcopy_qemufile_src; + } + /* Update and cache the current channel */ + rs->f =3D next; + rs->postcopy_channel =3D channel; + + /* + * If channel switched, reset last_sent_block since the old sent b= lock + * may not be on the same channel. + */ + rs->last_sent_block =3D NULL; + + trace_postcopy_preempt_switch_channel(channel); + } + + trace_postcopy_preempt_send_host_page(pss->block->idstr, pss->page); +} + +/* We need to make sure rs->f always points to the default channel elsewhe= re */ +static void postcopy_preempt_reset_channel(RAMState *rs) +{ + if (migrate_postcopy_preempt() && migration_in_postcopy()) { + rs->postcopy_channel =3D RAM_CHANNEL_PRECOPY; + rs->f =3D migrate_get_current()->to_dst_file; + trace_postcopy_preempt_reset_channel(); + } +} + /** * ram_save_host_page: save a whole host page * @@ -2211,7 +2406,16 @@ static int ram_save_host_page(RAMState *rs, PageSear= chStatus *pss) return 0; } =20 + if (migrate_postcopy_preempt() && migration_in_postcopy()) { + postcopy_preempt_choose_channel(rs, pss); + } + do { + if (postcopy_needs_preempt(rs, pss)) { + postcopy_do_preempt(rs, pss); + break; + } + /* Check the pages is dirty and if it is send it */ if (migration_bitmap_clear_dirty(rs, pss->block, pss->page)) { tmppages =3D ram_save_target_page(rs, pss); @@ -2235,6 +2439,19 @@ static int ram_save_host_page(RAMState *rs, PageSear= chStatus *pss) /* The offset we leave with is the min boundary of host page and block= */ pss->page =3D MIN(pss->page, hostpage_boundary); =20 + /* + * When with postcopy preempt mode, flush the data as soon as possible= for + * postcopy requests, because we've already sent a whole huge page, so= the + * dst node should already have enough resource to atomically filling = in + * the current missing page. + * + * More importantly, when using separate postcopy channel, we must do + * explicit flush or it won't flush until the buffer is full. + */ + if (migrate_postcopy_preempt() && pss->postcopy_requested) { + qemu_fflush(rs->f); + } + res =3D ram_save_release_protection(rs, pss, start_page); return (res < 0 ? res : pages); } @@ -2276,8 +2493,17 @@ static int ram_find_and_save_block(RAMState *rs) found =3D get_queued_page(rs, &pss); =20 if (!found) { - /* priority queue empty, so just search for something dirty */ - found =3D find_dirty_block(rs, &pss, &again); + /* + * Recover previous precopy ramblock/offset if postcopy has + * preempted precopy. Otherwise find the next dirty bit. + */ + if (postcopy_preempt_triggered(rs)) { + postcopy_preempt_restore(rs, &pss); + found =3D true; + } else { + /* priority queue empty, so just search for something dirt= y */ + found =3D find_dirty_block(rs, &pss, &again); + } } =20 if (found) { @@ -2405,6 +2631,8 @@ static void ram_state_reset(RAMState *rs) rs->last_page =3D 0; rs->last_version =3D ram_list.version; rs->xbzrle_enabled =3D false; + postcopy_preempt_reset(rs); + rs->postcopy_channel =3D RAM_CHANNEL_PRECOPY; } =20 #define MAX_WAIT 50 /* ms, half buffered_file limit */ @@ -3043,6 +3271,8 @@ static int ram_save_iterate(QEMUFile *f, void *opaque) } qemu_mutex_unlock(&rs->bitmap_mutex); =20 + postcopy_preempt_reset_channel(rs); + /* * Must occur before EOS (or any QEMUFile operation) * because of RDMA protocol. @@ -3112,6 +3342,8 @@ static int ram_save_complete(QEMUFile *f, void *opaqu= e) ram_control_after_iterate(f, RAM_CONTROL_FINISH); } =20 + postcopy_preempt_reset_channel(rs); + if (ret >=3D 0) { multifd_send_sync_main(rs->f); qemu_put_be64(f, RAM_SAVE_FLAG_EOS); @@ -3194,11 +3426,13 @@ static int load_xbzrle(QEMUFile *f, ram_addr_t addr= , void *host) * @mis: the migration incoming state pointer * @f: QEMUFile where to read the data from * @flags: Page flags (mostly to see if it's a continuation of previous bl= ock) + * @channel: the channel we're using */ static inline RAMBlock *ram_block_from_stream(MigrationIncomingState *mis, - QEMUFile *f, int flags) + QEMUFile *f, int flags, + int channel) { - RAMBlock *block =3D mis->last_recv_block; + RAMBlock *block =3D mis->last_recv_block[channel]; char id[256]; uint8_t len; =20 @@ -3225,7 +3459,7 @@ static inline RAMBlock *ram_block_from_stream(Migrati= onIncomingState *mis, return NULL; } =20 - mis->last_recv_block =3D block; + mis->last_recv_block[channel] =3D block; =20 return block; } @@ -3679,7 +3913,7 @@ int ram_load_postcopy(QEMUFile *f, int channel) trace_ram_load_postcopy_loop((uint64_t)addr, flags); if (flags & (RAM_SAVE_FLAG_ZERO | RAM_SAVE_FLAG_PAGE | RAM_SAVE_FLAG_COMPRESS_PAGE)) { - block =3D ram_block_from_stream(mis, f, flags); + block =3D ram_block_from_stream(mis, f, flags, channel); if (!block) { ret =3D -EINVAL; break; @@ -3930,7 +4164,7 @@ static int ram_load_precopy(QEMUFile *f) =20 if (flags & (RAM_SAVE_FLAG_ZERO | RAM_SAVE_FLAG_PAGE | RAM_SAVE_FLAG_COMPRESS_PAGE | RAM_SAVE_FLAG_XBZRLE)) { - RAMBlock *block =3D ram_block_from_stream(mis, f, flags); + RAMBlock *block =3D ram_block_from_stream(mis, f, flags, RAM_C= HANNEL_PRECOPY); =20 host =3D host_from_ram_block_offset(block, addr); /* diff --git a/migration/trace-events b/migration/trace-events index 4474a76614..b1155d9da0 100644 --- a/migration/trace-events +++ b/migration/trace-events @@ -111,6 +111,12 @@ ram_load_complete(int ret, uint64_t seq_iter) "exit_co= de %d seq iteration %" PRI ram_write_tracking_ramblock_start(const char *block_id, size_t page_size, = void *addr, size_t length) "%s: page_size: %zu addr: %p length: %zu" ram_write_tracking_ramblock_stop(const char *block_id, size_t page_size, v= oid *addr, size_t length) "%s: page_size: %zu addr: %p length: %zu" unqueue_page(char *block, uint64_t offset, bool dirty) "ramblock '%s' offs= et 0x%"PRIx64" dirty %d" +postcopy_preempt_triggered(char *str, unsigned long page) "during sending = ramblock %s offset 0x%lx" +postcopy_preempt_restored(char *str, unsigned long page) "ramblock %s offs= et 0x%lx" +postcopy_preempt_hit(char *str, uint64_t offset) "ramblock %s offset 0x%"P= RIx64 +postcopy_preempt_send_host_page(char *str, uint64_t offset) "ramblock %s o= ffset 0x%"PRIx64 +postcopy_preempt_switch_channel(int channel) "%d" +postcopy_preempt_reset_channel(void) "" =20 # multifd.c multifd_new_send_channel_async(uint8_t id) "channel %u" @@ -176,6 +182,7 @@ migration_thread_low_pending(uint64_t pending) "%" PRIu= 64 migrate_transferred(uint64_t tranferred, uint64_t time_spent, uint64_t ban= dwidth, uint64_t size) "transferred %" PRIu64 " time_spent %" PRIu64 " band= width %" PRIu64 " max_size %" PRId64 process_incoming_migration_co_end(int ret, int ps) "ret=3D%d postcopy-stat= e=3D%d" process_incoming_migration_co_postcopy_end_main(void) "" +postcopy_preempt_enabled(bool value) "%d" =20 # channel.c migration_set_incoming_channel(void *ioc, const char *ioctype) "ioc=3D%p i= octype=3D%s" --=20 2.32.0 From nobody Mon Feb 9 00:30:16 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1646126856762458.72148090831115; Tue, 1 Mar 2022 01:27:36 -0800 (PST) Received: from localhost ([::1]:43048 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nOyn8-0006iF-Sw for importer@patchew.org; Tue, 01 Mar 2022 04:27:35 -0500 Received: from eggs.gnu.org ([209.51.188.92]:54834) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nOy3c-0004R0-AM for qemu-devel@nongnu.org; Tue, 01 Mar 2022 03:40:33 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]:29181) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nOy3a-0008Kh-9f for qemu-devel@nongnu.org; Tue, 01 Mar 2022 03:40:32 -0500 Received: from mail-pf1-f198.google.com (mail-pf1-f198.google.com [209.85.210.198]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-497-RAr-iqF-M7aErH-lKRaK2Q-1; Tue, 01 Mar 2022 03:40:28 -0500 Received: by mail-pf1-f198.google.com with SMTP id 62-20020a621541000000b004f110fdb1aeso9381026pfv.13 for ; Tue, 01 Mar 2022 00:40:28 -0800 (PST) Received: from localhost.localdomain ([94.177.118.144]) by smtp.gmail.com with ESMTPSA id l1-20020a17090aec0100b001bc6d8bb27dsm1439987pjy.37.2022.03.01.00.40.24 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Tue, 01 Mar 2022 00:40:26 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1646124029; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=vDW0WR/YILw7vbC/woY+9lmdtY42QRLuti/BpXnPTr8=; b=Qq8vO92LCp3rEb0IjczHK8BwhwMKdh9WXRhWe4i5IwQ7CPTzMVWJFn143s2hktztpmdYZu PJj8M8fgsH6XAcI/S1zIVQ4TeNFl2/NDRNhU1dM4NnO5+/JWts2+GW583P/UEn/CXlaw99 JxjyxEc196Pv3srj9mPuLXye+eX/Yzc= X-MC-Unique: RAr-iqF-M7aErH-lKRaK2Q-1 X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=vDW0WR/YILw7vbC/woY+9lmdtY42QRLuti/BpXnPTr8=; b=YcM9CFLpDA3tOqrC9BwIooL28NrXr4HO8r4hQ0H8yEGYffExv1CijzKPuLqyv5xTgo Q6i8QDEz8C1Byc+dSSrkTpIl6PzhNmn98AYBsnNXkAXTuE7V+EzsWTiFTFJDHHC5PvvC 8o/0qfOVKDoYBaE0MYNj6fvH4PEOm1DKNBpF+ZWSM4GW1lRYC3bCJmR3aFkVuYdWJ9iS x325rcMzc8SGucdoD07GHrsfJt5bpk318up586nidJDjOrUW5xmep0a4UAHmc5MY8dKC sSaQR9xsByvAgLYMw0BzVtlF4WQIjhbGg5YewZQ/J0laXJOJlKPXh+p6iYveuMJGeMbW SNIA== X-Gm-Message-State: AOAM532uHApoCMFoLmLpgpVNN6+ImH3ZM0b14bgn+Dz1b31SmPfo9L50 AnjwZBCb2e2mR7xAWKK72+qJTMdYtlChCiTfwjupUexxOV6yUogVPaQfhyQatO+s3oqVCijrCXh Wnmz51Xf0hlvwt01FG3EWJeDqb5NQInOxWeEmF5Sz2JgENXSrPM5Va0RVSy/ucV6b X-Received: by 2002:a17:90a:5302:b0:1b9:ba0a:27e5 with SMTP id x2-20020a17090a530200b001b9ba0a27e5mr20438249pjh.91.1646124027504; Tue, 01 Mar 2022 00:40:27 -0800 (PST) X-Google-Smtp-Source: ABdhPJzZmThyYMXmNROFLEjD7sgE0XxQM4+KKwqGvmdKpr8PFTA82glYwZ+u9lVFB5ZA4EgAH4YO6g== X-Received: by 2002:a17:90a:5302:b0:1b9:ba0a:27e5 with SMTP id x2-20020a17090a530200b001b9ba0a27e5mr20438213pjh.91.1646124026975; Tue, 01 Mar 2022 00:40:26 -0800 (PST) From: Peter Xu To: qemu-devel@nongnu.org Subject: [PATCH v2 19/25] migration: Postcopy recover with preempt enabled Date: Tue, 1 Mar 2022 16:39:19 +0800 Message-Id: <20220301083925.33483-20-peterx@redhat.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20220301083925.33483-1-peterx@redhat.com> References: <20220301083925.33483-1-peterx@redhat.com> MIME-Version: 1.0 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=peterx@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=peterx@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -28 X-Spam_score: -2.9 X-Spam_bar: -- X-Spam_report: (-2.9 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.082, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H5=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Juan Quintela , "Dr . David Alan Gilbert" , peterx@redhat.com, Leonardo Bras Soares Passos Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZM-MESSAGEID: 1646126858834100001 Content-Type: text/plain; charset="utf-8" To allow postcopy recovery, the ram fast load (preempt-only) dest QEMU thre= ad needs similar handling on fault tolerance. When ram_load_postcopy() fails, instead of stopping the thread it halts with a semaphore, preparing to be kicked again when recovery is detected. A mutex is introduced to make sure there's no concurrent operation upon the socket. To make it simple, the fast ram load thread will take the mutex du= ring its whole procedure, and only release it if it's paused. The fast-path soc= ket will be properly released by the main loading thread safely when there's network failures during postcopy with that mutex held. Reviewed-by: Dr. David Alan Gilbert Signed-off-by: Peter Xu --- migration/migration.c | 27 +++++++++++++++++++++++---- migration/migration.h | 19 +++++++++++++++++++ migration/postcopy-ram.c | 24 ++++++++++++++++++++++-- migration/qemu-file.c | 27 +++++++++++++++++++++++++++ migration/qemu-file.h | 1 + migration/savevm.c | 21 +++++++++++++++++++-- migration/trace-events | 2 ++ 7 files changed, 113 insertions(+), 8 deletions(-) diff --git a/migration/migration.c b/migration/migration.c index d20db04097..69778cab23 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -215,9 +215,11 @@ void migration_object_init(void) current_incoming->postcopy_remote_fds =3D g_array_new(FALSE, TRUE, sizeof(struct PostCopyFD)); qemu_mutex_init(¤t_incoming->rp_mutex); + qemu_mutex_init(¤t_incoming->postcopy_prio_thread_mutex); qemu_event_init(¤t_incoming->main_thread_load_event, false); qemu_sem_init(¤t_incoming->postcopy_pause_sem_dst, 0); qemu_sem_init(¤t_incoming->postcopy_pause_sem_fault, 0); + qemu_sem_init(¤t_incoming->postcopy_pause_sem_fast_load, 0); qemu_mutex_init(¤t_incoming->page_request_mutex); current_incoming->page_requested =3D g_tree_new(page_request_addr_cmp); =20 @@ -697,9 +699,9 @@ static bool postcopy_try_recover(void) =20 /* * Here, we only wake up the main loading thread (while the - * fault thread will still be waiting), so that we can receive + * rest threads will still be waiting), so that we can receive * commands from source now, and answer it if needed. The - * fault thread will be woken up afterwards until we are sure + * rest threads will be woken up afterwards until we are sure * that source is ready to reply to page requests. */ qemu_sem_post(&mis->postcopy_pause_sem_dst); @@ -3466,6 +3468,18 @@ static MigThrError postcopy_pause(MigrationState *s) qemu_file_shutdown(file); qemu_fclose(file); =20 + /* + * Do the same to postcopy fast path socket too if there is. No + * locking needed because no racer as long as we do this before se= tting + * status to paused. + */ + if (s->postcopy_qemufile_src) { + migration_ioc_unregister_yank_from_file(s->postcopy_qemufile_s= rc); + qemu_file_shutdown(s->postcopy_qemufile_src); + qemu_fclose(s->postcopy_qemufile_src); + s->postcopy_qemufile_src =3D NULL; + } + migrate_set_state(&s->state, s->state, MIGRATION_STATUS_POSTCOPY_PAUSED); =20 @@ -3521,8 +3535,13 @@ static MigThrError migration_detect_error(MigrationS= tate *s) return MIG_THR_ERR_FATAL; } =20 - /* Try to detect any file errors */ - ret =3D qemu_file_get_error_obj(s->to_dst_file, &local_error); + /* + * Try to detect any file errors. Note that postcopy_qemufile_src will + * be NULL when postcopy preempt is not enabled. + */ + ret =3D qemu_file_get_error_obj_any(s->to_dst_file, + s->postcopy_qemufile_src, + &local_error); if (!ret) { /* Everything is fine */ assert(!local_error); diff --git a/migration/migration.h b/migration/migration.h index b8aacfe3af..91f845e9e4 100644 --- a/migration/migration.h +++ b/migration/migration.h @@ -118,6 +118,18 @@ struct MigrationIncomingState { /* Postcopy priority thread is used to receive postcopy requested page= s */ QemuThread postcopy_prio_thread; bool postcopy_prio_thread_created; + /* + * Used to sync between the ram load main thread and the fast ram load + * thread. It protects postcopy_qemufile_dst, which is the postcopy + * fast channel. + * + * The ram fast load thread will take it mostly for the whole lifecycle + * because it needs to continuously read data from the channel, and + * it'll only release this mutex if postcopy is interrupted, so that + * the ram load main thread will take this mutex over and properly + * release the broken channel. + */ + QemuMutex postcopy_prio_thread_mutex; /* * An array of temp host huge pages to be used, one for each postcopy * channel. @@ -147,6 +159,13 @@ struct MigrationIncomingState { /* notify PAUSED postcopy incoming migrations to try to continue */ QemuSemaphore postcopy_pause_sem_dst; QemuSemaphore postcopy_pause_sem_fault; + /* + * This semaphore is used to allow the ram fast load thread (only when + * postcopy preempt is enabled) fall into sleep when there's network + * interruption detected. When the recovery is done, the main load + * thread will kick the fast ram load thread using this semaphore. + */ + QemuSemaphore postcopy_pause_sem_fast_load; =20 /* List of listening socket addresses */ SocketAddressList *socket_address_list; diff --git a/migration/postcopy-ram.c b/migration/postcopy-ram.c index df0c02f729..e20305a9e2 100644 --- a/migration/postcopy-ram.c +++ b/migration/postcopy-ram.c @@ -1580,6 +1580,15 @@ int postcopy_preempt_setup(MigrationState *s, Error = **errp) return 0; } =20 +static void postcopy_pause_ram_fast_load(MigrationIncomingState *mis) +{ + trace_postcopy_pause_fast_load(); + qemu_mutex_unlock(&mis->postcopy_prio_thread_mutex); + qemu_sem_wait(&mis->postcopy_pause_sem_fast_load); + qemu_mutex_lock(&mis->postcopy_prio_thread_mutex); + trace_postcopy_pause_fast_load_continued(); +} + void *postcopy_preempt_thread(void *opaque) { MigrationIncomingState *mis =3D opaque; @@ -1592,11 +1601,22 @@ void *postcopy_preempt_thread(void *opaque) qemu_sem_post(&mis->thread_sync_sem); =20 /* Sending RAM_SAVE_FLAG_EOS to terminate this thread */ - ret =3D ram_load_postcopy(mis->postcopy_qemufile_dst, RAM_CHANNEL_POST= COPY); + qemu_mutex_lock(&mis->postcopy_prio_thread_mutex); + while (1) { + ret =3D ram_load_postcopy(mis->postcopy_qemufile_dst, RAM_CHANNEL_= POSTCOPY); + /* If error happened, go into recovery routine */ + if (ret) { + postcopy_pause_ram_fast_load(mis); + } else { + /* We're done */ + break; + } + } + qemu_mutex_unlock(&mis->postcopy_prio_thread_mutex); =20 rcu_unregister_thread(); =20 trace_postcopy_preempt_thread_exit(); =20 - return ret =3D=3D 0 ? NULL : (void *)-1; + return NULL; } diff --git a/migration/qemu-file.c b/migration/qemu-file.c index 1479cddad9..397652f0ba 100644 --- a/migration/qemu-file.c +++ b/migration/qemu-file.c @@ -139,6 +139,33 @@ int qemu_file_get_error_obj(QEMUFile *f, Error **errp) return f->last_error; } =20 +/* + * Get last error for either stream f1 or f2 with optional Error*. + * The error returned (non-zero) can be either from f1 or f2. + * + * If any of the qemufile* is NULL, then skip the check on that file. + * + * When there is no error on both qemufile, zero is returned. + */ +int qemu_file_get_error_obj_any(QEMUFile *f1, QEMUFile *f2, Error **errp) +{ + int ret =3D 0; + + if (f1) { + ret =3D qemu_file_get_error_obj(f1, errp); + /* If there's already error detected, return */ + if (ret) { + return ret; + } + } + + if (f2) { + ret =3D qemu_file_get_error_obj(f2, errp); + } + + return ret; +} + /* * Set the last error for stream f with optional Error* */ diff --git a/migration/qemu-file.h b/migration/qemu-file.h index 3f36d4dc8c..2564e5e1c7 100644 --- a/migration/qemu-file.h +++ b/migration/qemu-file.h @@ -156,6 +156,7 @@ void qemu_file_update_transfer(QEMUFile *f, int64_t len= ); void qemu_file_set_rate_limit(QEMUFile *f, int64_t new_rate); int64_t qemu_file_get_rate_limit(QEMUFile *f); int qemu_file_get_error_obj(QEMUFile *f, Error **errp); +int qemu_file_get_error_obj_any(QEMUFile *f1, QEMUFile *f2, Error **errp); void qemu_file_set_error_obj(QEMUFile *f, int ret, Error *err); void qemu_file_set_error(QEMUFile *f, int ret); int qemu_file_shutdown(QEMUFile *f); diff --git a/migration/savevm.c b/migration/savevm.c index 254aa78234..24b69a1008 100644 --- a/migration/savevm.c +++ b/migration/savevm.c @@ -2152,6 +2152,13 @@ static int loadvm_postcopy_handle_resume(MigrationIn= comingState *mis) */ qemu_sem_post(&mis->postcopy_pause_sem_fault); =20 + if (migrate_postcopy_preempt()) { + /* The channel should already be setup again; make sure of it */ + assert(mis->postcopy_qemufile_dst); + /* Kick the fast ram load thread too */ + qemu_sem_post(&mis->postcopy_pause_sem_fast_load); + } + return 0; } =20 @@ -2607,6 +2614,16 @@ static bool postcopy_pause_incoming(MigrationIncomin= gState *mis) mis->to_src_file =3D NULL; qemu_mutex_unlock(&mis->rp_mutex); =20 + if (mis->postcopy_qemufile_dst) { + qemu_file_shutdown(mis->postcopy_qemufile_dst); + /* Take the mutex to make sure the fast ram load thread halted */ + qemu_mutex_lock(&mis->postcopy_prio_thread_mutex); + migration_ioc_unregister_yank_from_file(mis->postcopy_qemufile_dst= ); + qemu_fclose(mis->postcopy_qemufile_dst); + mis->postcopy_qemufile_dst =3D NULL; + qemu_mutex_unlock(&mis->postcopy_prio_thread_mutex); + } + migrate_set_state(&mis->state, MIGRATION_STATUS_POSTCOPY_ACTIVE, MIGRATION_STATUS_POSTCOPY_PAUSED); =20 @@ -2634,8 +2651,8 @@ retry: while (true) { section_type =3D qemu_get_byte(f); =20 - if (qemu_file_get_error(f)) { - ret =3D qemu_file_get_error(f); + ret =3D qemu_file_get_error_obj_any(f, mis->postcopy_qemufile_dst,= NULL); + if (ret) { break; } =20 diff --git a/migration/trace-events b/migration/trace-events index b1155d9da0..dfe36a3340 100644 --- a/migration/trace-events +++ b/migration/trace-events @@ -270,6 +270,8 @@ mark_postcopy_blocktime_begin(uint64_t addr, void *dd, = uint32_t time, int cpu, i mark_postcopy_blocktime_end(uint64_t addr, void *dd, uint32_t time, int af= fected_cpu) "addr: 0x%" PRIx64 ", dd: %p, time: %u, affected_cpu: %d" postcopy_pause_fault_thread(void) "" postcopy_pause_fault_thread_continued(void) "" +postcopy_pause_fast_load(void) "" +postcopy_pause_fast_load_continued(void) "" postcopy_ram_fault_thread_entry(void) "" postcopy_ram_fault_thread_exit(void) "" postcopy_ram_fault_thread_fds_core(int baseufd, int quitfd) "ufd: %d quitf= d: %d" --=20 2.32.0 From nobody Mon Feb 9 00:30:16 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1646127219131643.5413342918933; Tue, 1 Mar 2022 01:33:39 -0800 (PST) Received: from localhost ([::1]:50738 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nOyt0-0003n0-4H for importer@patchew.org; Tue, 01 Mar 2022 04:33:38 -0500 Received: from eggs.gnu.org ([209.51.188.92]:54878) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nOy3f-0004UK-UV for qemu-devel@nongnu.org; Tue, 01 Mar 2022 03:40:38 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:39314) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nOy3c-0008LW-WA for qemu-devel@nongnu.org; Tue, 01 Mar 2022 03:40:35 -0500 Received: from mail-pg1-f198.google.com (mail-pg1-f198.google.com [209.85.215.198]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-336-wIocJMx_NLiKreYtTefViw-1; Tue, 01 Mar 2022 03:40:31 -0500 Received: by mail-pg1-f198.google.com with SMTP id v4-20020a63f844000000b003745fd0919aso8149136pgj.20 for ; Tue, 01 Mar 2022 00:40:31 -0800 (PST) Received: from localhost.localdomain ([94.177.118.144]) by smtp.gmail.com with ESMTPSA id l1-20020a17090aec0100b001bc6d8bb27dsm1439987pjy.37.2022.03.01.00.40.27 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Tue, 01 Mar 2022 00:40:29 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1646124032; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=DfpYQNmkkLjgEV2zMsOyOlP/tyky1HmQMXBErffeIdA=; b=K2pj38NHPM9eLePouDlKO//RI9FZuLqSLJa4K2qU+hC/20Gn2RvBfRyrclg1HzEQQux1t/ 6SHhQcn4zi3ej6HkMjgbgy3+wkvEeyIZcLjYnAtv7aRpSTJmGeXMxEbBvk3G5LzqhuAgvc gTYqCXdk2v4Km2XUpFnjIxHYdgmt+WU= X-MC-Unique: wIocJMx_NLiKreYtTefViw-1 X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=DfpYQNmkkLjgEV2zMsOyOlP/tyky1HmQMXBErffeIdA=; b=UFz2NO27UE5xrxJCfbeBiuT/ydL8uRLNhfYAuQ4+m/2CcO7CXwKkYvHt1bmpA+oBfk 44U9AQfiItfD+50+col6oLZ9+X1owT4qTmJWSY5kG5H9M+hF8VsO5jzMZfRuqf2TgzFm cikXrAZONuRXXy4cV/4ZEHGZzN66gvgIB46MP7piQ8fsRBdK5M4aPw8wf+EqvtZtVAWs OblgspECLKRqklUZ+3oM1J48Eg6DurXOV75Tof2CcCMYZgjW4UlSt5wWXhtCXUzAgjS/ 6TZjQvMjU3bLuWDP58ZVe8c2A/rC9bHaS2Phxb0l9eBeZ54ee3q9cuogulIOtbqq/I3y Kd7w== X-Gm-Message-State: AOAM5328i048I0lNlxdYrNC+yt8OccOKnZ+17sWRzB8YKSRkLKBWMeWY 7kM4FvuogUgA43NoGh94WsEiriXHbmSbXU82tppj+K6ltDoMAQPHu767j4Xi9I7DRDTInJRqqBx sVblUc5SfRs3/94sxM5wO8YavLPLzusoPnrAdIgpJ9TqiluJtQxsRu31VRcfBjQna X-Received: by 2002:a63:141d:0:b0:342:585e:a519 with SMTP id u29-20020a63141d000000b00342585ea519mr21155626pgl.257.1646124030009; Tue, 01 Mar 2022 00:40:30 -0800 (PST) X-Google-Smtp-Source: ABdhPJyYXnCXopx62EaONaQsW6G98Jyhk8Lzj37txHe879nzabfXjwdMjaZ2eRISxwvEUDHKRVxkEA== X-Received: by 2002:a63:141d:0:b0:342:585e:a519 with SMTP id u29-20020a63141d000000b00342585ea519mr21155601pgl.257.1646124029576; Tue, 01 Mar 2022 00:40:29 -0800 (PST) From: Peter Xu To: qemu-devel@nongnu.org Subject: [PATCH v2 20/25] migration: Create the postcopy preempt channel asynchronously Date: Tue, 1 Mar 2022 16:39:20 +0800 Message-Id: <20220301083925.33483-21-peterx@redhat.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20220301083925.33483-1-peterx@redhat.com> References: <20220301083925.33483-1-peterx@redhat.com> MIME-Version: 1.0 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=peterx@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=peterx@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -28 X-Spam_score: -2.9 X-Spam_bar: -- X-Spam_report: (-2.9 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.082, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H5=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Juan Quintela , "Dr . David Alan Gilbert" , peterx@redhat.com, Leonardo Bras Soares Passos Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZM-MESSAGEID: 1646127220144100001 Content-Type: text/plain; charset="utf-8" This patch allows the postcopy preempt channel to be created asynchronously. The benefit is that when the connection is slow, we won't take the BQL (and potentially block all things like QMP) for a long time without releasing. A function postcopy_preempt_wait_channel() is introduced, allowing the migration thread to be able to wait on the channel creation. The channel is always created by the main thread, in which we'll kick a new semaphore to tell the migration thread that the channel has created. We'll need to wait for the new channel in two places: (1) when there's a new postcopy migration that is starting, or (2) when there's a postcopy migration to resume. For the start of migration, we don't need to wait for this channel until when we want to start postcopy, aka, postcopy_start(). We'll fail the migration if we found that the channel creation failed (which should probably not happen at all in 99% of the cases, because the main channel is using the same network topology). For a postcopy recovery, we'll need to wait in postcopy_pause(). In that case if the channel creation failed, we can't fail the migration or we'll crash the VM, instead we keep in PAUSED state, waiting for yet another recovery. Signed-off-by: Peter Xu --- migration/migration.c | 16 ++++++++++++ migration/migration.h | 7 +++++ migration/postcopy-ram.c | 56 +++++++++++++++++++++++++++++++--------- migration/postcopy-ram.h | 1 + 4 files changed, 68 insertions(+), 12 deletions(-) diff --git a/migration/migration.c b/migration/migration.c index 69778cab23..78e1e6bfb9 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -3016,6 +3016,12 @@ static int postcopy_start(MigrationState *ms) int64_t bandwidth =3D migrate_max_postcopy_bandwidth(); bool restart_block =3D false; int cur_state =3D MIGRATION_STATUS_ACTIVE; + + if (postcopy_preempt_wait_channel(ms)) { + migrate_set_state(&ms->state, ms->state, MIGRATION_STATUS_FAILED); + return -1; + } + if (!migrate_pause_before_switchover()) { migrate_set_state(&ms->state, MIGRATION_STATUS_ACTIVE, MIGRATION_STATUS_POSTCOPY_ACTIVE); @@ -3497,6 +3503,14 @@ static MigThrError postcopy_pause(MigrationState *s) if (s->state =3D=3D MIGRATION_STATUS_POSTCOPY_RECOVER) { /* Woken up by a recover procedure. Give it a shot */ =20 + if (postcopy_preempt_wait_channel(s)) { + /* + * Preempt enabled, and new channel create failed; loop + * back to wait for another recovery. + */ + continue; + } + /* * Firstly, let's wake up the return path now, with a new * return path channel. @@ -4356,6 +4370,7 @@ static void migration_instance_finalize(Object *obj) qemu_sem_destroy(&ms->postcopy_pause_sem); qemu_sem_destroy(&ms->postcopy_pause_rp_sem); qemu_sem_destroy(&ms->rp_state.rp_sem); + qemu_sem_destroy(&ms->postcopy_qemufile_src_sem); error_free(ms->error); } =20 @@ -4402,6 +4417,7 @@ static void migration_instance_init(Object *obj) qemu_sem_init(&ms->rp_state.rp_sem, 0); qemu_sem_init(&ms->rate_limit_sem, 0); qemu_sem_init(&ms->wait_unplug_sem, 0); + qemu_sem_init(&ms->postcopy_qemufile_src_sem, 0); qemu_mutex_init(&ms->qemu_file_lock); } =20 diff --git a/migration/migration.h b/migration/migration.h index 91f845e9e4..f898b8547a 100644 --- a/migration/migration.h +++ b/migration/migration.h @@ -219,6 +219,13 @@ struct MigrationState { QEMUFile *to_dst_file; /* Postcopy specific transfer channel */ QEMUFile *postcopy_qemufile_src; + /* + * It is posted when the preempt channel is established. Note: this is + * used for both the start or recover of a postcopy migration. We'll + * post to this sem every time a new preempt channel is created in the + * main thread, and we keep post() and wait() in pair. + */ + QemuSemaphore postcopy_qemufile_src_sem; QIOChannelBuffer *bioc; /* * Protects to_dst_file/from_dst_file pointers. We need to make sure = we diff --git a/migration/postcopy-ram.c b/migration/postcopy-ram.c index e20305a9e2..3ead5b1b3c 100644 --- a/migration/postcopy-ram.c +++ b/migration/postcopy-ram.c @@ -1552,10 +1552,50 @@ bool postcopy_preempt_new_channel(MigrationIncoming= State *mis, QEMUFile *file) return true; } =20 -int postcopy_preempt_setup(MigrationState *s, Error **errp) +static void +postcopy_preempt_send_channel_new(QIOTask *task, gpointer opaque) { - QIOChannel *ioc; + MigrationState *s =3D opaque; + QIOChannel *ioc =3D QIO_CHANNEL(qio_task_get_source(task)); + Error *local_err =3D NULL; + + if (qio_task_propagate_error(task, &local_err)) { + /* Something wrong happened.. */ + object_unref(OBJECT(ioc)); + migrate_set_error(migrate_get_current(), local_err); + error_free(local_err); + } else { + migration_ioc_register_yank(ioc); + s->postcopy_qemufile_src =3D qemu_fopen_channel_output(ioc); + trace_postcopy_preempt_new_channel(); + } + + /* + * Kick the waiter in all cases. The waiter should check upon + * postcopy_qemufile_src to know whether it failed or not. + */ + qemu_sem_post(&s->postcopy_qemufile_src_sem); +} =20 +/* Returns 0 if channel established, -1 for error. */ +int postcopy_preempt_wait_channel(MigrationState *s) +{ + /* If preempt not enabled, no need to wait */ + if (!migrate_postcopy_preempt()) { + return 0; + } + + /* + * We need the postcopy preempt channel to be established before + * starting doing anything. + */ + qemu_sem_wait(&s->postcopy_qemufile_src_sem); + + return s->postcopy_qemufile_src ? 0 : -1; +} + +int postcopy_preempt_setup(MigrationState *s, Error **errp) +{ if (!migrate_postcopy_preempt()) { return 0; } @@ -1566,16 +1606,8 @@ int postcopy_preempt_setup(MigrationState *s, Error = **errp) return -1; } =20 - ioc =3D socket_send_channel_create_sync(errp); - - if (ioc =3D=3D NULL) { - return -1; - } - - migration_ioc_register_yank(ioc); - s->postcopy_qemufile_src =3D qemu_fopen_channel_output(ioc); - - trace_postcopy_preempt_new_channel(); + /* Kick an async task to connect */ + socket_send_channel_create(postcopy_preempt_send_channel_new, s); =20 return 0; } diff --git a/migration/postcopy-ram.h b/migration/postcopy-ram.h index 34b1080cde..6147bf7d1d 100644 --- a/migration/postcopy-ram.h +++ b/migration/postcopy-ram.h @@ -192,5 +192,6 @@ enum PostcopyChannels { =20 bool postcopy_preempt_new_channel(MigrationIncomingState *mis, QEMUFile *f= ile); int postcopy_preempt_setup(MigrationState *s, Error **errp); +int postcopy_preempt_wait_channel(MigrationState *s); =20 #endif --=20 2.32.0 From nobody Mon Feb 9 00:30:16 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1646125432000432.391367391744; Tue, 1 Mar 2022 01:03:52 -0800 (PST) Received: from localhost ([::1]:39106 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nOyQB-0000jd-8r for importer@patchew.org; Tue, 01 Mar 2022 04:03:51 -0500 Received: from eggs.gnu.org ([209.51.188.92]:54906) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nOy3i-0004UR-Ca for qemu-devel@nongnu.org; Tue, 01 Mar 2022 03:40:38 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]:21787) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nOy3f-0008M3-DH for qemu-devel@nongnu.org; Tue, 01 Mar 2022 03:40:37 -0500 Received: from mail-pl1-f199.google.com (mail-pl1-f199.google.com [209.85.214.199]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-75-yrVb9k40MzyIktwuapXW9g-1; Tue, 01 Mar 2022 03:40:33 -0500 Received: by mail-pl1-f199.google.com with SMTP id c12-20020a170902848c00b0015025f53e9cso4545630plo.7 for ; Tue, 01 Mar 2022 00:40:33 -0800 (PST) Received: from localhost.localdomain ([94.177.118.144]) by smtp.gmail.com with ESMTPSA id l1-20020a17090aec0100b001bc6d8bb27dsm1439987pjy.37.2022.03.01.00.40.29 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Tue, 01 Mar 2022 00:40:31 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1646124034; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=UApSsKIRKkSZ48pbdKt2A0O84YoHKmN0sdhMzMfmg3k=; b=VkJVgEBZq9DCBms3dZbUedz41iU02a8Kgkp4/tJmlJgCPn3PCZKO+Dfpw+DWTzc2UqpPls ZOGmAvMHoSK/RHcRsJ6H4RLOt2ApCYcQzrJreurYrdLpE9dGe1Wz9Qlh/vMBy4pdkcsoJG mTzz2ydyUS4IfayEG5CB8V6EBF09ne4= X-MC-Unique: yrVb9k40MzyIktwuapXW9g-1 X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=UApSsKIRKkSZ48pbdKt2A0O84YoHKmN0sdhMzMfmg3k=; b=XnqOMyTL6Nv/vJT21Mv1J++tXdZ3+/XJ5ZOd/5HWKZNfZY+Petx+7sXCDs29c4geET 5oTqlilXJwH0fvWorqdW59pcKZyjmcRqFRidK1TIL6uMXrzTXT4ylLGoYfFI0Mt+mRJb Qqwamt1+4/qi/lre+X/e9n8oxSFF6WAswGWvEGwhr3v2UtBoSvB2Cn+ywXTNbbRilgsr Wr90ANujFwtJ/KDczMXcmx+6bLBcqu7sjJ/EWGTqZ/8i+6VA43ovIn9F9Gz/nQmAvdm2 WeEh4ovbXi4RTiArcIOZQRwSVl6nYHCUBaz2K9wNXAJQIKCbyOW2zrOXuiBCa7lyVQPh nuxQ== X-Gm-Message-State: AOAM53078kLUzMJ+o0YwrxolGBsbKgTjTflX6+HViu8QJPonhU+XQZGK 3CbzVv1+lhqdlHYhve/b5IGsUtTjCftCsvwxGLqBSAGy7e7j06fbMPOIv9X23uXY04eLBQNVzc3 dJw7FgCfb0v2SOeuTIYawoZTdS2FlnfARiPicREHqysGi74NqqRibklTBQAL8Evyr X-Received: by 2002:a63:e543:0:b0:373:efe3:6f46 with SMTP id z3-20020a63e543000000b00373efe36f46mr21052856pgj.183.1646124032567; Tue, 01 Mar 2022 00:40:32 -0800 (PST) X-Google-Smtp-Source: ABdhPJxnHBnzGV3M7F/woRV0OD3df33i693Nj7v4rkONr7iDlD7UB7PSlAzE4xKS+4ZHTvHlW/ILjA== X-Received: by 2002:a63:e543:0:b0:373:efe3:6f46 with SMTP id z3-20020a63e543000000b00373efe36f46mr21052831pgj.183.1646124032201; Tue, 01 Mar 2022 00:40:32 -0800 (PST) From: Peter Xu To: qemu-devel@nongnu.org Subject: [PATCH v2 21/25] migration: Parameter x-postcopy-preempt-break-huge Date: Tue, 1 Mar 2022 16:39:21 +0800 Message-Id: <20220301083925.33483-22-peterx@redhat.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20220301083925.33483-1-peterx@redhat.com> References: <20220301083925.33483-1-peterx@redhat.com> MIME-Version: 1.0 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=peterx@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=peterx@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -28 X-Spam_score: -2.9 X-Spam_bar: -- X-Spam_report: (-2.9 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.082, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H5=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Juan Quintela , "Dr . David Alan Gilbert" , peterx@redhat.com, Leonardo Bras Soares Passos Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZM-MESSAGEID: 1646125433973100001 Content-Type: text/plain; charset="utf-8" Add a parameter that can conditionally disable the "break sending huge page" behavior in postcopy preemption. By default it's enabled. It should only be used for debugging purposes, and we should never remove the "x-" prefix. Signed-off-by: Peter Xu --- migration/migration.c | 2 ++ migration/migration.h | 7 +++++++ migration/ram.c | 7 +++++++ 3 files changed, 16 insertions(+) diff --git a/migration/migration.c b/migration/migration.c index 78e1e6bfb9..cd4a150202 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -4325,6 +4325,8 @@ static Property migration_properties[] =3D { DEFINE_PROP_SIZE("announce-step", MigrationState, parameters.announce_step, DEFAULT_MIGRATE_ANNOUNCE_STEP), + DEFINE_PROP_BOOL("x-postcopy-preempt-break-huge", MigrationState, + postcopy_preempt_break_huge, true), =20 /* Migration capabilities */ DEFINE_PROP_MIG_CAP("x-xbzrle", MIGRATION_CAPABILITY_XBZRLE), diff --git a/migration/migration.h b/migration/migration.h index f898b8547a..6ee520642f 100644 --- a/migration/migration.h +++ b/migration/migration.h @@ -340,6 +340,13 @@ struct MigrationState { bool send_configuration; /* Whether we send section footer during migration */ bool send_section_footer; + /* + * Whether we allow break sending huge pages when postcopy preempt is + * enabled. When disabled, we won't interrupt precopy within sending a + * host huge page, which is the old behavior of vanilla postcopy. + * NOTE: this parameter is ignored if postcopy preempt is not enabled. + */ + bool postcopy_preempt_break_huge; =20 /* Needed by postcopy-pause state */ QemuSemaphore postcopy_pause_sem; diff --git a/migration/ram.c b/migration/ram.c index 53dfd9be38..ede8aaac01 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -2266,11 +2266,18 @@ static int ram_save_target_page(RAMState *rs, PageS= earchStatus *pss) =20 static bool postcopy_needs_preempt(RAMState *rs, PageSearchStatus *pss) { + MigrationState *ms =3D migrate_get_current(); + /* Not enabled eager preempt? Then never do that. */ if (!migrate_postcopy_preempt()) { return false; } =20 + /* If the user explicitly disabled breaking of huge page, skip */ + if (!ms->postcopy_preempt_break_huge) { + return false; + } + /* If the ramblock we're sending is a small page? Never bother. */ if (qemu_ram_pagesize(pss->block) =3D=3D TARGET_PAGE_SIZE) { return false; --=20 2.32.0 From nobody Mon Feb 9 00:30:16 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1646125819172136.67538279498137; Tue, 1 Mar 2022 01:10:19 -0800 (PST) Received: from localhost ([::1]:47066 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nOyWP-0006Nz-VS for importer@patchew.org; Tue, 01 Mar 2022 04:10:18 -0500 Received: from eggs.gnu.org ([209.51.188.92]:54922) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nOy3k-0004Wm-6A for qemu-devel@nongnu.org; Tue, 01 Mar 2022 03:40:43 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]:30694) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nOy3i-0008MX-6Q for qemu-devel@nongnu.org; Tue, 01 Mar 2022 03:40:39 -0500 Received: from mail-pj1-f71.google.com (mail-pj1-f71.google.com [209.85.216.71]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-295-R362sVgZPxOYLIshTpRjHg-1; Tue, 01 Mar 2022 03:40:36 -0500 Received: by mail-pj1-f71.google.com with SMTP id m3-20020a17090a3f8300b001bd33af55f2so1212328pjc.1 for ; Tue, 01 Mar 2022 00:40:36 -0800 (PST) Received: from localhost.localdomain ([94.177.118.144]) by smtp.gmail.com with ESMTPSA id l1-20020a17090aec0100b001bc6d8bb27dsm1439987pjy.37.2022.03.01.00.40.32 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Tue, 01 Mar 2022 00:40:34 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1646124037; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=wWGELbbBUsyM61ubBs61dKQWxxrQsZk2X7zAss01FCA=; b=fvtk6ZSb0xUOsk07tnlEl/72sQ+GTB0hk6ltMUYpeCBIPeh2BHsjEKurT9MgfyQWRfpEzK /tbRrlx9DvmOK5CLCzQ2QKbbFgUFgdkhe0WbGWCFAHz5W6ub64PVUTWL4msYrs2pNBnp6D +DHV2VIUUrCDrbd/sFvRb84mNkQ0Ou8= X-MC-Unique: R362sVgZPxOYLIshTpRjHg-1 X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=wWGELbbBUsyM61ubBs61dKQWxxrQsZk2X7zAss01FCA=; b=5R0yx5lu7o4l+KDjn6dmq45gMkPiAxpZ8z2YP9rh9j24dcDTikCWWp6UXexBjpTty9 bAD3eqffNcL95OpoDLQYkstEtfWMtGLcWyPmYldpaJ0UQQ2Nio22RJEdmBObL/i/E4VU NErLaznr9XQzY4Lp1qoPfZmOOodSS7Gmsk0wdxKVr0NQ23pqxz748jeSqzZ0Ij6DYJ65 DuPeALG0wzO5ES0ZnAdASCeNYEBPFjOi+I7m5aJpDB8G2AHW4LqfYGYkjxveLFjSf7tZ 7t8cWUo2YwhPFV4Z7hcIfWpzqsiLh+itoKH2WkvzfsDLKFWvhj2xeztmEuuclreEiiCW SQLA== X-Gm-Message-State: AOAM530Wg24uTD1PeuSSpDTmUnRBQNdz+pRg087B8Z4xkc5kB8Sgk7FC 817KVyUUX9p0YYB/FPAEftN1I2cwGjfjJ0V51syUynCMdNdx+JpnZH7lBOeF0U0A8WpCfv/7kMk TfzlpRJeQ4UbyjHnXKI5sTeN9p/lnPga3hN6rQ4Kewv2/QadUChXZSUP8u0xWH3X5 X-Received: by 2002:a63:944:0:b0:374:5324:eea1 with SMTP id 65-20020a630944000000b003745324eea1mr20964856pgj.366.1646124035467; Tue, 01 Mar 2022 00:40:35 -0800 (PST) X-Google-Smtp-Source: ABdhPJwmYhh6E7VjGY5Qbh+0HMxF0pxaLwc9NPrkQHwFuN1M+Z2f7/bfbUOej+a9RVBF0v6xFDqRAQ== X-Received: by 2002:a63:944:0:b0:374:5324:eea1 with SMTP id 65-20020a630944000000b003745324eea1mr20964832pgj.366.1646124035093; Tue, 01 Mar 2022 00:40:35 -0800 (PST) From: Peter Xu To: qemu-devel@nongnu.org Subject: [PATCH v2 22/25] migration: Add helpers to detect TLS capability Date: Tue, 1 Mar 2022 16:39:22 +0800 Message-Id: <20220301083925.33483-23-peterx@redhat.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20220301083925.33483-1-peterx@redhat.com> References: <20220301083925.33483-1-peterx@redhat.com> MIME-Version: 1.0 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=peterx@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=peterx@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -28 X-Spam_score: -2.9 X-Spam_bar: -- X-Spam_report: (-2.9 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.082, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H5=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Juan Quintela , "Dr . David Alan Gilbert" , peterx@redhat.com, Leonardo Bras Soares Passos Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZM-MESSAGEID: 1646125821168100001 Content-Type: text/plain; charset="utf-8" Add migrate_tls_enabled() to detect whether TLS is configured. Add migrate_channel_requires_tls() to detect whether the specific channel requires TLS. No functional change intended. Signed-off-by: Peter Xu --- migration/channel.c | 10 ++-------- migration/migration.c | 17 +++++++++++++++++ migration/migration.h | 4 ++++ migration/multifd.c | 7 +------ 4 files changed, 24 insertions(+), 14 deletions(-) diff --git a/migration/channel.c b/migration/channel.c index c4fc000a1a..85ac053275 100644 --- a/migration/channel.c +++ b/migration/channel.c @@ -38,10 +38,7 @@ void migration_channel_process_incoming(QIOChannel *ioc) trace_migration_set_incoming_channel( ioc, object_get_typename(OBJECT(ioc))); =20 - if (s->parameters.tls_creds && - *s->parameters.tls_creds && - !object_dynamic_cast(OBJECT(ioc), - TYPE_QIO_CHANNEL_TLS)) { + if (migrate_channel_requires_tls(ioc)) { migration_tls_channel_process_incoming(s, ioc, &local_err); } else { migration_ioc_register_yank(ioc); @@ -71,10 +68,7 @@ void migration_channel_connect(MigrationState *s, ioc, object_get_typename(OBJECT(ioc)), hostname, error); =20 if (!error) { - if (s->parameters.tls_creds && - *s->parameters.tls_creds && - !object_dynamic_cast(OBJECT(ioc), - TYPE_QIO_CHANNEL_TLS)) { + if (migrate_channel_requires_tls(ioc)) { migration_tls_channel_connect(s, ioc, hostname, &error); =20 if (!error) { diff --git a/migration/migration.c b/migration/migration.c index cd4a150202..f30bad982c 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -49,6 +49,7 @@ #include "trace.h" #include "exec/target_page.h" #include "io/channel-buffer.h" +#include "io/channel-tls.h" #include "migration/colo.h" #include "hw/boards.h" #include "hw/qdev-properties.h" @@ -4246,6 +4247,22 @@ void migration_global_dump(Monitor *mon) ms->clear_bitmap_shift); } =20 +bool migrate_tls_enabled(void) +{ + MigrationState *s =3D migrate_get_current(); + + return s->parameters.tls_creds && *s->parameters.tls_creds; +} + +bool migrate_channel_requires_tls(QIOChannel *ioc) +{ + if (!migrate_tls_enabled()) { + return false; + } + + return !object_dynamic_cast(OBJECT(ioc), TYPE_QIO_CHANNEL_TLS); +} + #define DEFINE_PROP_MIG_CAP(name, x) \ DEFINE_PROP_BOOL(name, MigrationState, enabled_capabilities[x], false) =20 diff --git a/migration/migration.h b/migration/migration.h index 6ee520642f..8b9ad7fe31 100644 --- a/migration/migration.h +++ b/migration/migration.h @@ -436,6 +436,10 @@ bool migrate_use_events(void); bool migrate_postcopy_blocktime(void); bool migrate_background_snapshot(void); bool migrate_postcopy_preempt(void); +/* Whether TLS is enabled for migration? */ +bool migrate_tls_enabled(void); +/* Whether the QIO channel requires further TLS handshake? */ +bool migrate_channel_requires_tls(QIOChannel *ioc); =20 /* Sending on the return path - generic and then for each message type */ void migrate_send_rp_shut(MigrationIncomingState *mis, diff --git a/migration/multifd.c b/migration/multifd.c index 180586dcde..46dfcbfa1d 100644 --- a/migration/multifd.c +++ b/migration/multifd.c @@ -784,16 +784,11 @@ static bool multifd_channel_connect(MultiFDSendParams= *p, QIOChannel *ioc, Error *error) { - MigrationState *s =3D migrate_get_current(); - trace_multifd_set_outgoing_channel( ioc, object_get_typename(OBJECT(ioc)), p->tls_hostname, error); =20 if (!error) { - if (s->parameters.tls_creds && - *s->parameters.tls_creds && - !object_dynamic_cast(OBJECT(ioc), - TYPE_QIO_CHANNEL_TLS)) { + if (migrate_channel_requires_tls(ioc)) { multifd_tls_channel_connect(p, ioc, &error); if (!error) { /* --=20 2.32.0 From nobody Mon Feb 9 00:30:16 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1646126198162395.22786862695637; Tue, 1 Mar 2022 01:16:38 -0800 (PST) Received: from localhost ([::1]:53636 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nOycX-0002bb-2T for importer@patchew.org; Tue, 01 Mar 2022 04:16:37 -0500 Received: from eggs.gnu.org ([209.51.188.92]:54940) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nOy3m-0004Ws-6l for qemu-devel@nongnu.org; Tue, 01 Mar 2022 03:40:43 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]:40830) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nOy3k-0008N3-QV for qemu-devel@nongnu.org; Tue, 01 Mar 2022 03:40:41 -0500 Received: from mail-pj1-f72.google.com (mail-pj1-f72.google.com [209.85.216.72]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-593-JWGdUapgOiWGeOwO1JGG3g-1; Tue, 01 Mar 2022 03:40:39 -0500 Received: by mail-pj1-f72.google.com with SMTP id u11-20020a17090ae00b00b001bc4cef20f1so8024540pjy.9 for ; Tue, 01 Mar 2022 00:40:39 -0800 (PST) Received: from localhost.localdomain ([94.177.118.144]) by smtp.gmail.com with ESMTPSA id l1-20020a17090aec0100b001bc6d8bb27dsm1439987pjy.37.2022.03.01.00.40.35 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Tue, 01 Mar 2022 00:40:37 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1646124040; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=oT6l+V3jJuwCFvnRE26WZY8M0dO/7ULSHBa48JNUl4g=; b=hDp7IdyPUPNIlTpXPfZ37ZeAIc6p7bUy77BvAZ4U4eqZE49CuYhFxBdjs7YX7iUXTQtnoH 0S5G+yoTckY5JzUIlAWfs8pHXwUWOwzlm97V854XtqgHUXFLt7pxwJ0SJvUwWBEo7R0qAl dv+E+P8dJdLhf3fr+hf9GCoU6GooMVI= X-MC-Unique: JWGdUapgOiWGeOwO1JGG3g-1 X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=oT6l+V3jJuwCFvnRE26WZY8M0dO/7ULSHBa48JNUl4g=; b=PTXOD46G6Wlrq9KEEhuKuSM/IyK71qTcghaUaVoOcFJjRI/vyCmCbRKhb+ZZOiJCVs 1iOIZMe+d1QM1ifQgdDuvTGfF7weneAC11uSp1j710Uime6FCdasqZjva3wMD2OopRjE uSirPd0LZ5pwHLOtvQ9TI8jtFsYb1SJQoCKMktOHX8b0df6cG2FLnNIWhaZ5wA4O/nAI DwEeLUOPzdQYDN7M2k6IuQV5P3RzboF7EBzHeIgOxcHyZ7/ijmEcncoVCSg6J9sB+3dJ r1RCRDRPBTyc0+DTj4wYlbWL7xbuOnBYrJ6KgeCY4JWMoXebyHaMg3hrdynCG01ntzfW +aRw== X-Gm-Message-State: AOAM531aqqrN+jHA8a+DeTor837v/s5wlOPsC7mOfudDnNbz9YMeiNQq VkVzaUW1so26vH7HaNJe62JXka9e3vQN0h9p73VNIs8vb+Ekq06sZxX/Pq0CdoogAKjp5i+4OBN 0xYCTcwm8urCzveJC3RcoBd3FTmQx7gXR9yyfEDtnJiT/4MSTR13QK7go2R4cKjso X-Received: by 2002:a17:902:8d81:b0:14d:964b:6186 with SMTP id v1-20020a1709028d8100b0014d964b6186mr24343436plo.108.1646124038239; Tue, 01 Mar 2022 00:40:38 -0800 (PST) X-Google-Smtp-Source: ABdhPJyfnAJVEv2+Z05118KGkOpVDfW4Zds/LLw6Er+hvDrddjhS9XhC8yktyDjoMpThwbvxcSb4oA== X-Received: by 2002:a17:902:8d81:b0:14d:964b:6186 with SMTP id v1-20020a1709028d8100b0014d964b6186mr24343412plo.108.1646124037869; Tue, 01 Mar 2022 00:40:37 -0800 (PST) From: Peter Xu To: qemu-devel@nongnu.org Subject: [PATCH v2 23/25] migration: Fail postcopy preempt with TLS for now Date: Tue, 1 Mar 2022 16:39:23 +0800 Message-Id: <20220301083925.33483-24-peterx@redhat.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20220301083925.33483-1-peterx@redhat.com> References: <20220301083925.33483-1-peterx@redhat.com> MIME-Version: 1.0 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=peterx@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=peterx@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -28 X-Spam_score: -2.9 X-Spam_bar: -- X-Spam_report: (-2.9 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.082, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H5=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Juan Quintela , "Dr . David Alan Gilbert" , peterx@redhat.com, Leonardo Bras Soares Passos Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZM-MESSAGEID: 1646126200181100001 Content-Type: text/plain; charset="utf-8" The support is not yet there. Temporarily fail it properly when starting postcopy until it's supported. Fail at postcopy-start still allows the user to proceed with e.g. pure tls precopy even if postcopy-ram is set. Signed-off-by: Peter Xu --- migration/migration.c | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/migration/migration.c b/migration/migration.c index f30bad982c..95cfc483c9 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -1802,6 +1802,12 @@ void qmp_migrate_start_postcopy(Error **errp) " started"); return; } + + if (migrate_postcopy_preempt() && migrate_tls_enabled()) { + error_setg(errp, "Postcopy preemption does not support TLS yet"); + return; + } + /* * we don't error if migration has finished since that would be racy * with issuing this command. --=20 2.32.0 From nobody Mon Feb 9 00:30:16 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1646125319174886.4221787761702; Tue, 1 Mar 2022 01:01:59 -0800 (PST) Received: from localhost ([::1]:36800 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nOyOL-0007RG-42 for importer@patchew.org; Tue, 01 Mar 2022 04:01:57 -0500 Received: from eggs.gnu.org ([209.51.188.92]:55040) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nOy4T-0005do-TP for qemu-devel@nongnu.org; Tue, 01 Mar 2022 03:41:26 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:45642) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nOy4S-0008RE-AG for qemu-devel@nongnu.org; Tue, 01 Mar 2022 03:41:25 -0500 Received: from mail-pl1-f197.google.com (mail-pl1-f197.google.com [209.85.214.197]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-534-1jZdpb3DOqmCIHDlRBKHfA-1; Tue, 01 Mar 2022 03:40:42 -0500 Received: by mail-pl1-f197.google.com with SMTP id n12-20020a170902e54c00b0015011534fa3so5951695plf.9 for ; Tue, 01 Mar 2022 00:40:42 -0800 (PST) Received: from localhost.localdomain ([94.177.118.144]) by smtp.gmail.com with ESMTPSA id l1-20020a17090aec0100b001bc6d8bb27dsm1439987pjy.37.2022.03.01.00.40.38 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Tue, 01 Mar 2022 00:40:40 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1646124083; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=lerOqNBKq7I7gR+BYtu9C6JMejBVtKQ6XmvGsCpkV5o=; b=HCGAMOhq4edx7AUbhL0HGGe4KL9eDqzWALnyaW4ZyhXlbnGjgOqBXHufQK7DN7dJ7iAXCj Z/DHE9Swe40yHuOjmx/V3aDl2YVS6dXUaaSZwxEQaFx5qZt0hHEgg1ZstF//N8iAQCX0QF 6y26+7sQKWsvFvumLS2O6Dfa2OWlPYU= X-MC-Unique: 1jZdpb3DOqmCIHDlRBKHfA-1 X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=lerOqNBKq7I7gR+BYtu9C6JMejBVtKQ6XmvGsCpkV5o=; b=eNCILbq0QShANfBdpP5YUsMtkFvGDCD2gvdFRdaoZu5WEtXHZ3G2XA2mFgpL7jtY9+ o29mgbxLoqcvTq4/zWxwiJUyFw7MfXqmeEgPK9KMWmRc2bMhSUBWah9oyfMgkOf+HXZ7 lmGMvF5W7NEo5Mb1E/jn82xbSlg0Yktmbbs/bDna1mpkvK3VUuHT32VVlhZSr3Txpi5w /NxYeevL4H56NHjCmLs9LU2K39aDrNdl4qMVWN79frQUIheHoGxwubL20iHNTpi01Riz 43kxiUhR4pEEbFdPSnmY5thqYa/jlWpmgmNqhJycgY1lh1Uz38kLz+O7nKE1gMybpeJN y03Q== X-Gm-Message-State: AOAM530FjpHsp373x7xjs/a7/4tlmWMOxWoCahqUl63wtZA8SMWckw+j yU9Lu9mUlHWwJ89GL/Ykl5yCiyjweGj4f4fayTYGC45al8XcpUY6Nh6Gp66SQxZDVZq0exjqPSZ HXSzSg39nV7mMFwzbc906MhzUZjd5IoV5JmnvljYfuzvEupgrVfETPCWnLcPduJng X-Received: by 2002:a17:90a:a42:b0:1bd:1985:5182 with SMTP id o60-20020a17090a0a4200b001bd19855182mr15033208pjo.5.1646124041204; Tue, 01 Mar 2022 00:40:41 -0800 (PST) X-Google-Smtp-Source: ABdhPJxeH5hX2wAhlpz6Qx6fKgwDGI5j0/YCiD2LQQLPuGJl4PzaoNSD1eAXMTdifUhKJ5X+8LNEHg== X-Received: by 2002:a17:90a:a42:b0:1bd:1985:5182 with SMTP id o60-20020a17090a0a4200b001bd19855182mr15033185pjo.5.1646124040772; Tue, 01 Mar 2022 00:40:40 -0800 (PST) From: Peter Xu To: qemu-devel@nongnu.org Subject: [PATCH v2 24/25] tests: Add postcopy preempt test Date: Tue, 1 Mar 2022 16:39:24 +0800 Message-Id: <20220301083925.33483-25-peterx@redhat.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20220301083925.33483-1-peterx@redhat.com> References: <20220301083925.33483-1-peterx@redhat.com> MIME-Version: 1.0 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=peterx@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=peterx@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -28 X-Spam_score: -2.9 X-Spam_bar: -- X-Spam_report: (-2.9 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.082, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H5=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Juan Quintela , "Dr . David Alan Gilbert" , peterx@redhat.com, Leonardo Bras Soares Passos Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZM-MESSAGEID: 1646125321959100001 Content-Type: text/plain; charset="utf-8" Two tests are added: a normal postcopy preempt test, and a recovery test. Signed-off-by: Peter Xu --- tests/qtest/migration-test.c | 41 ++++++++++++++++++++++++++++++++++-- 1 file changed, 39 insertions(+), 2 deletions(-) diff --git a/tests/qtest/migration-test.c b/tests/qtest/migration-test.c index 7b42f6fd90..09a9ce4401 100644 --- a/tests/qtest/migration-test.c +++ b/tests/qtest/migration-test.c @@ -470,6 +470,7 @@ typedef struct { */ bool hide_stderr; bool use_shmem; + bool postcopy_preempt; /* only launch the target process */ bool only_target; /* Use dirty ring if true; dirty logging otherwise */ @@ -663,6 +664,8 @@ static int migrate_postcopy_prepare(QTestState **from_p= tr, MigrateStart *args) { g_autofree char *uri =3D g_strdup_printf("unix:%s/migsocket", tmpfs); + /* NOTE: args will be freed in test_migrate_start(), cache it */ + bool postcopy_preempt =3D args->postcopy_preempt; QTestState *from, *to; =20 if (test_migrate_start(&from, &to, uri, args)) { @@ -673,6 +676,11 @@ static int migrate_postcopy_prepare(QTestState **from_= ptr, migrate_set_capability(to, "postcopy-ram", true); migrate_set_capability(to, "postcopy-blocktime", true); =20 + if (postcopy_preempt) { + migrate_set_capability(from, "postcopy-preempt", true); + migrate_set_capability(to, "postcopy-preempt", true); + } + /* We want to pick a speed slow enough that the test completes * quickly, but that it doesn't complete precopy even on a slow * machine, so also set the downtime. @@ -719,13 +727,29 @@ static void test_postcopy(void) migrate_postcopy_complete(from, to); } =20 -static void test_postcopy_recovery(void) +static void test_postcopy_preempt(void) +{ + MigrateStart *args =3D migrate_start_new(); + QTestState *from, *to; + + args->postcopy_preempt =3D true; + + if (migrate_postcopy_prepare(&from, &to, args)) { + return; + } + migrate_postcopy_start(from, to); + migrate_postcopy_complete(from, to); +} + +/* @preempt: whether to use postcopy-preempt */ +static void test_postcopy_recovery(bool preempt) { MigrateStart *args =3D migrate_start_new(); QTestState *from, *to; g_autofree char *uri =3D NULL; =20 args->hide_stderr =3D true; + args->postcopy_preempt =3D preempt; =20 if (migrate_postcopy_prepare(&from, &to, args)) { return; @@ -781,6 +805,16 @@ static void test_postcopy_recovery(void) migrate_postcopy_complete(from, to); } =20 +static void test_postcopy_recovery_normal(void) +{ + test_postcopy_recovery(false); +} + +static void test_postcopy_recovery_preempt(void) +{ + test_postcopy_recovery(true); +} + static void test_baddest(void) { MigrateStart *args =3D migrate_start_new(); @@ -1458,7 +1492,10 @@ int main(int argc, char **argv) module_call_init(MODULE_INIT_QOM); =20 qtest_add_func("/migration/postcopy/unix", test_postcopy); - qtest_add_func("/migration/postcopy/recovery", test_postcopy_recovery); + qtest_add_func("/migration/postcopy/recovery", test_postcopy_recovery_= normal); + qtest_add_func("/migration/postcopy/preempt/unix", test_postcopy_preem= pt); + qtest_add_func("/migration/postcopy/preempt/recovery", + test_postcopy_recovery_preempt); qtest_add_func("/migration/bad_dest", test_baddest); qtest_add_func("/migration/precopy/unix", test_precopy_unix); qtest_add_func("/migration/precopy/tcp", test_precopy_tcp); --=20 2.32.0 From nobody Mon Feb 9 00:30:16 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1646126482317891.1677690283341; Tue, 1 Mar 2022 01:21:22 -0800 (PST) Received: from localhost ([::1]:33986 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nOyh7-0000Ip-NT for importer@patchew.org; Tue, 01 Mar 2022 04:21:21 -0500 Received: from eggs.gnu.org ([209.51.188.92]:54978) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nOy3s-0004gb-It for qemu-devel@nongnu.org; Tue, 01 Mar 2022 03:40:48 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:43724) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nOy3q-0008O1-UZ for qemu-devel@nongnu.org; Tue, 01 Mar 2022 03:40:48 -0500 Received: from mail-pl1-f197.google.com (mail-pl1-f197.google.com [209.85.214.197]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-586-iH3__BlnMwGUSHw--ZPczg-1; Tue, 01 Mar 2022 03:40:45 -0500 Received: by mail-pl1-f197.google.com with SMTP id l6-20020a170903120600b0014f43ba55f3so5958090plh.11 for ; Tue, 01 Mar 2022 00:40:45 -0800 (PST) Received: from localhost.localdomain ([94.177.118.144]) by smtp.gmail.com with ESMTPSA id l1-20020a17090aec0100b001bc6d8bb27dsm1439987pjy.37.2022.03.01.00.40.41 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Tue, 01 Mar 2022 00:40:43 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1646124046; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=iScz+bjSHJXZSsec6ODclid8MpYXvGhjXV9oo6c4Wa0=; b=eHb5tjKgpAZ9gK/huvzpa8ZM7KVHKax5Jv5rMF0YHSS35rmKeA2DlH73zsLe6Aci4LHE0Q qD2pqMBnqygf3y0dFPYp43FrBdgwmnLQ33e3vqMjSoh3TXNmk0RDLP/j+m0DMHKAzLJd7w 3mdd50SeQHKToBbak0tTsVpmiWdfV4k= X-MC-Unique: iH3__BlnMwGUSHw--ZPczg-1 X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=iScz+bjSHJXZSsec6ODclid8MpYXvGhjXV9oo6c4Wa0=; b=wOsfBQjqNj2SYxh/av9QmabGJAqE/t12Mlsr6e8t6/gtFpeTmZYU0h++EkW95mm151 JnELpiaoT9t4GWonENgcEwZFszSzhBhMuT35IXB/RwTMA/5lnrjX3YvocLUjjJDVtyj+ 5rYcqtd/Jc5qF4g98RjrHHjobvcumScnWEpQPjVn4XV/OPlwX/Z6r3yWZrCZE8rUxHlY oVSsasNjNDi6BB7qk+Nggc7pdN8ppMiHCM8OkDQ2krvvbA51aXfDxQzFSR0hRBEFoFHJ yY4hO/avCa5M2OnS0V1WwNYVRSb/j9Mz2Ysmc8gZn7Rbd/Wkxe9VpY+iFhlxNcr/m4Bl EDBQ== X-Gm-Message-State: AOAM531QA4wW7NJw8og5QpqN6rsCeTFJIlS4pHnwrQGzfe9ioPuV/st6 q3kJ9LIDjbHl7O2FPduxg5nxsKDe1/1s10ts+FwyPQ+NZ/entLJn79vJcaYbX4i3T3rxEAoDFXy YY4hUjBdwVcn6aUcW7yyHgyDEy/ySUlB759ShKxPAFF5tV1iXTJkWheGeQJuo7Xrm X-Received: by 2002:a05:6a00:be5:b0:4e1:9050:1e16 with SMTP id x37-20020a056a000be500b004e190501e16mr26316100pfu.78.1646124044208; Tue, 01 Mar 2022 00:40:44 -0800 (PST) X-Google-Smtp-Source: ABdhPJzSW1XpCPK3wLmJ212KpT+hOGu0zWtTtDF8mWy8PI/laH4dL3iF7J+yu/z7guawxE46IYNy+Q== X-Received: by 2002:a05:6a00:be5:b0:4e1:9050:1e16 with SMTP id x37-20020a056a000be500b004e190501e16mr26316067pfu.78.1646124043766; Tue, 01 Mar 2022 00:40:43 -0800 (PST) From: Peter Xu To: qemu-devel@nongnu.org Subject: [PATCH v2 25/25] tests: Pass in MigrateStart** into test_migrate_start() Date: Tue, 1 Mar 2022 16:39:25 +0800 Message-Id: <20220301083925.33483-26-peterx@redhat.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20220301083925.33483-1-peterx@redhat.com> References: <20220301083925.33483-1-peterx@redhat.com> MIME-Version: 1.0 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=peterx@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=peterx@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -28 X-Spam_score: -2.9 X-Spam_bar: -- X-Spam_report: (-2.9 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.082, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H5=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Juan Quintela , "Dr . David Alan Gilbert" , peterx@redhat.com, Leonardo Bras Soares Passos Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZM-MESSAGEID: 1646126484364100001 Content-Type: text/plain; charset="utf-8" test_migrate_start() will release the MigrateStart structure that passed in, however that's not super clear to the caller because after the call returned the pointer can still be referenced by the callers. It can easily be a source of use-after-free. Let's pass in a double pointer of that, then we can safely clear the pointer for the caller after the struct is released. Signed-off-by: Peter Xu Reviewed-by: Dr. David Alan Gilbert --- tests/qtest/migration-test.c | 27 +++++++++++++++------------ 1 file changed, 15 insertions(+), 12 deletions(-) diff --git a/tests/qtest/migration-test.c b/tests/qtest/migration-test.c index 09a9ce4401..67f0601988 100644 --- a/tests/qtest/migration-test.c +++ b/tests/qtest/migration-test.c @@ -496,7 +496,7 @@ static void migrate_start_destroy(MigrateStart *args) } =20 static int test_migrate_start(QTestState **from, QTestState **to, - const char *uri, MigrateStart *args) + const char *uri, MigrateStart **pargs) { g_autofree gchar *arch_source =3D NULL; g_autofree gchar *arch_target =3D NULL; @@ -508,6 +508,7 @@ static int test_migrate_start(QTestState **from, QTestS= tate **to, g_autofree char *shmem_path =3D NULL; const char *arch =3D qtest_get_arch(); const char *machine_opts =3D NULL; + MigrateStart *args =3D *pargs; const char *memory_size; int ret =3D 0; =20 @@ -622,6 +623,8 @@ static int test_migrate_start(QTestState **from, QTestS= tate **to, =20 out: migrate_start_destroy(args); + /* This tells the caller that this structure is gone */ + *pargs =3D NULL; return ret; } =20 @@ -668,7 +671,7 @@ static int migrate_postcopy_prepare(QTestState **from_p= tr, bool postcopy_preempt =3D args->postcopy_preempt; QTestState *from, *to; =20 - if (test_migrate_start(&from, &to, uri, args)) { + if (test_migrate_start(&from, &to, uri, &args)) { return -1; } =20 @@ -822,7 +825,7 @@ static void test_baddest(void) =20 args->hide_stderr =3D true; =20 - if (test_migrate_start(&from, &to, "tcp:127.0.0.1:0", args)) { + if (test_migrate_start(&from, &to, "tcp:127.0.0.1:0", &args)) { return; } migrate_qmp(from, "tcp:127.0.0.1:0", "{}"); @@ -838,7 +841,7 @@ static void test_precopy_unix_common(bool dirty_ring) =20 args->use_dirty_ring =3D dirty_ring; =20 - if (test_migrate_start(&from, &to, uri, args)) { + if (test_migrate_start(&from, &to, uri, &args)) { return; } =20 @@ -926,7 +929,7 @@ static void test_xbzrle(const char *uri) MigrateStart *args =3D migrate_start_new(); QTestState *from, *to; =20 - if (test_migrate_start(&from, &to, uri, args)) { + if (test_migrate_start(&from, &to, uri, &args)) { return; } =20 @@ -980,7 +983,7 @@ static void test_precopy_tcp(void) g_autofree char *uri =3D NULL; QTestState *from, *to; =20 - if (test_migrate_start(&from, &to, "tcp:127.0.0.1:0", args)) { + if (test_migrate_start(&from, &to, "tcp:127.0.0.1:0", &args)) { return; } =20 @@ -1025,7 +1028,7 @@ static void test_migrate_fd_proto(void) QDict *rsp; const char *error_desc; =20 - if (test_migrate_start(&from, &to, "defer", args)) { + if (test_migrate_start(&from, &to, "defer", &args)) { return; } =20 @@ -1105,7 +1108,7 @@ static void do_test_validate_uuid(MigrateStart *args,= bool should_fail) g_autofree char *uri =3D g_strdup_printf("unix:%s/migsocket", tmpfs); QTestState *from, *to; =20 - if (test_migrate_start(&from, &to, uri, args)) { + if (test_migrate_start(&from, &to, uri, &args)) { return; } =20 @@ -1197,7 +1200,7 @@ static void test_migrate_auto_converge(void) */ const int64_t expected_threshold =3D max_bandwidth * downtime_limit / = 1000; =20 - if (test_migrate_start(&from, &to, uri, args)) { + if (test_migrate_start(&from, &to, uri, &args)) { return; } =20 @@ -1266,7 +1269,7 @@ static void test_multifd_tcp(const char *method) QDict *rsp; g_autofree char *uri =3D NULL; =20 - if (test_migrate_start(&from, &to, "defer", args)) { + if (test_migrate_start(&from, &to, "defer", &args)) { return; } =20 @@ -1352,7 +1355,7 @@ static void test_multifd_tcp_cancel(void) =20 args->hide_stderr =3D true; =20 - if (test_migrate_start(&from, &to, "defer", args)) { + if (test_migrate_start(&from, &to, "defer", &args)) { return; } =20 @@ -1391,7 +1394,7 @@ static void test_multifd_tcp_cancel(void) args =3D migrate_start_new(); args->only_target =3D true; =20 - if (test_migrate_start(&from, &to2, "defer", args)) { + if (test_migrate_start(&from, &to2, "defer", &args)) { return; } =20 --=20 2.32.0