From nobody Mon Feb 9 07:55:43 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1648677063784778.2499715762716; Wed, 30 Mar 2022 14:51:03 -0700 (PDT) Received: from localhost ([::1]:56756 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nZgDW-0001Pp-Ns for importer@patchew.org; Wed, 30 Mar 2022 17:51:02 -0400 Received: from eggs.gnu.org ([209.51.188.92]:37934) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nZg2I-0005um-47 for qemu-devel@nongnu.org; Wed, 30 Mar 2022 17:39:26 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]:41560) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nZg2F-0005FU-Ew for qemu-devel@nongnu.org; Wed, 30 Mar 2022 17:39:25 -0400 Received: from mail-qt1-f200.google.com (mail-qt1-f200.google.com [209.85.160.200]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-619-FTPRZg8IOOibeltMZsNAbw-1; Wed, 30 Mar 2022 17:39:21 -0400 Received: by mail-qt1-f200.google.com with SMTP id f22-20020ac840d6000000b002dd4d87de21so18397441qtm.23 for ; Wed, 30 Mar 2022 14:39:21 -0700 (PDT) Received: from localhost.localdomain (cpec09435e3e0ee-cmc09435e3e0ec.cpe.net.cable.rogers.com. [99.241.198.116]) by smtp.gmail.com with ESMTPSA id a23-20020a05620a16d700b0067e98304705sm11306313qkn.89.2022.03.30.14.39.19 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Wed, 30 Mar 2022 14:39:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1648676362; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=oLh7AWTsxENQn04JHEx/vmWZbl1/dnGUdXevqLuUURg=; b=Q02ZODpeV7t/pT9uUOa2cG1l9SiFGjTwZPIqD730iNGfYhBUwadOtV9ITf/0JxrUvHlt9M 07qwbaWYtUuwGDgoDO0wdN6POyltOVtLBJgyIrE+JFyPwA1x8FEMlOQoOhHDUaAB0gFeyY 6dzlGnnRt8uvgJABnIchfSjuNJC3Urk= X-MC-Unique: FTPRZg8IOOibeltMZsNAbw-1 X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=oLh7AWTsxENQn04JHEx/vmWZbl1/dnGUdXevqLuUURg=; b=a0KL1PhFk4YnxycAcDvRCYyICZrKWZ4Uatl90/awG1QyGxeJf2CWO3V9OgX3ZQ4vGV 8IahB+v3X6+syDE8gtMvBpAgGLBXFztFAyUFJ3LvTi1kSE/rCWglr+egVsQOnWzJBjRR pR9Eb7gmbI4wDlpqh+RFWrhqc5mHo/qMTBGvGYMw//XJkajq2cMB8ZAjRWQvv8P4e+Ew JKfaGIlHww3HYyWnOJeUuEzuOEtsu6yrEdx2YvJEmUcTkira5ln13z4wiLQv+H43LDyw 9A5s9ZhfBY8viICxvWT9V3DmyESaoUp3FE4FZJIYGhozf10dSjFy+i0xV6eRB2rVvKej wbwg== X-Gm-Message-State: AOAM530uPIeIdFTwL1S8VqvbuyhE/Kza2gbZUEMJATen6pINh1Xlt9VJ SAjDDS2+aWE65sXCsq+/CaQ2so8IUfKQCHgaRU2Cfm5POlOwdgPifhyurxPAJZyDvOuDPydsV/g slHfZbHsRJh7+knihiOXN8/ZnRSbREZJ1LnYrqJx3hm0J5nx1aKWmyc9IFPt1YbW+ X-Received: by 2002:a05:620a:25cb:b0:67e:9ae8:1563 with SMTP id y11-20020a05620a25cb00b0067e9ae81563mr1282724qko.222.1648676360882; Wed, 30 Mar 2022 14:39:20 -0700 (PDT) X-Google-Smtp-Source: ABdhPJwB1CClX1MV4G0/JtrEKQiPqWnf+sIVlHwjpwP55ZyjqJ06vKzzJPFXDxcR6twNtGoKsSas9w== X-Received: by 2002:a05:620a:25cb:b0:67e:9ae8:1563 with SMTP id y11-20020a05620a25cb00b0067e9ae81563mr1282695qko.222.1648676360398; Wed, 30 Mar 2022 14:39:20 -0700 (PDT) From: Peter Xu To: qemu-devel@nongnu.org Subject: [PATCH v3 09/19] migration: Postcopy preemption preparation on channel creation Date: Wed, 30 Mar 2022 17:38:58 -0400 Message-Id: <20220330213908.26608-10-peterx@redhat.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20220330213908.26608-1-peterx@redhat.com> References: <20220330213908.26608-1-peterx@redhat.com> MIME-Version: 1.0 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=peterx@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=peterx@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -22 X-Spam_score: -2.3 X-Spam_bar: -- X-Spam_report: (-2.3 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.082, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01, URG_BIZ=0.573 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: "Dr . David Alan Gilbert" , "Daniel P . Berrange" , Leonardo Bras Soares Passos , peterx@redhat.com, Juan Quintela Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZM-MESSAGEID: 1648677064211100001 Content-Type: text/plain; charset="utf-8"; x-default="true" Create a new socket for postcopy to be prepared to send postcopy requested pages via this specific channel, so as to not get blocked by precopy pages. A new thread is also created on dest qemu to receive data from this new cha= nnel based on the ram_load_postcopy() routine. The ram_load_postcopy(POSTCOPY) branch and the thread has not started to function, and that'll be done in follow up patches. Cleanup the new sockets on both src/dst QEMUs, meanwhile look after the new thread too to make sure it'll be recycled properly. Signed-off-by: Peter Xu --- migration/migration.c | 62 +++++++++++++++++++++++---- migration/migration.h | 8 ++++ migration/postcopy-ram.c | 92 ++++++++++++++++++++++++++++++++++++++-- migration/postcopy-ram.h | 10 +++++ migration/ram.c | 25 ++++++++--- migration/ram.h | 4 +- migration/savevm.c | 20 ++++----- migration/socket.c | 22 +++++++++- migration/socket.h | 1 + migration/trace-events | 5 ++- 10 files changed, 218 insertions(+), 31 deletions(-) diff --git a/migration/migration.c b/migration/migration.c index 76e6ada524..01b882494d 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -321,6 +321,12 @@ void migration_incoming_state_destroy(void) mis->page_requested =3D NULL; } =20 + if (mis->postcopy_qemufile_dst) { + migration_ioc_unregister_yank_from_file(mis->postcopy_qemufile_dst= ); + qemu_fclose(mis->postcopy_qemufile_dst); + mis->postcopy_qemufile_dst =3D NULL; + } + yank_unregister_instance(MIGRATION_YANK_INSTANCE); } =20 @@ -714,15 +720,21 @@ void migration_fd_process_incoming(QEMUFile *f, Error= **errp) migration_incoming_process(); } =20 +static bool migration_needs_multiple_sockets(void) +{ + return migrate_use_multifd() || migrate_postcopy_preempt(); +} + void migration_ioc_process_incoming(QIOChannel *ioc, Error **errp) { MigrationIncomingState *mis =3D migration_incoming_get_current(); Error *local_err =3D NULL; bool start_migration; + QEMUFile *f; =20 if (!mis->from_src_file) { /* The first connection (multifd may have multiple) */ - QEMUFile *f =3D qemu_fopen_channel_input(ioc); + f =3D qemu_fopen_channel_input(ioc); =20 if (!migration_incoming_setup(f, errp)) { return; @@ -730,13 +742,18 @@ void migration_ioc_process_incoming(QIOChannel *ioc, = Error **errp) =20 /* * Common migration only needs one channel, so we can start - * right now. Multifd needs more than one channel, we wait. + * right now. Some features need more than one channel, we wait. */ - start_migration =3D !migrate_use_multifd(); + start_migration =3D !migration_needs_multiple_sockets(); } else { /* Multiple connections */ - assert(migrate_use_multifd()); - start_migration =3D multifd_recv_new_channel(ioc, &local_err); + assert(migration_needs_multiple_sockets()); + if (migrate_use_multifd()) { + start_migration =3D multifd_recv_new_channel(ioc, &local_err); + } else if (migrate_postcopy_preempt()) { + f =3D qemu_fopen_channel_input(ioc); + start_migration =3D postcopy_preempt_new_channel(mis, f); + } if (local_err) { error_propagate(errp, local_err); return; @@ -761,11 +778,20 @@ void migration_ioc_process_incoming(QIOChannel *ioc, = Error **errp) bool migration_has_all_channels(void) { MigrationIncomingState *mis =3D migration_incoming_get_current(); - bool all_channels; =20 - all_channels =3D multifd_recv_all_channels_created(); + if (!mis->from_src_file) { + return false; + } + + if (migrate_use_multifd()) { + return multifd_recv_all_channels_created(); + } + + if (migrate_postcopy_preempt()) { + return mis->postcopy_qemufile_dst !=3D NULL; + } =20 - return all_channels && mis->from_src_file !=3D NULL; + return true; } =20 /* @@ -1863,6 +1889,12 @@ static void migrate_fd_cleanup(MigrationState *s) qemu_fclose(tmp); } =20 + if (s->postcopy_qemufile_src) { + migration_ioc_unregister_yank_from_file(s->postcopy_qemufile_src); + qemu_fclose(s->postcopy_qemufile_src); + s->postcopy_qemufile_src =3D NULL; + } + assert(!migration_is_active(s)); =20 if (s->state =3D=3D MIGRATION_STATUS_CANCELLING) { @@ -3238,6 +3270,11 @@ static void migration_completion(MigrationState *s) qemu_savevm_state_complete_postcopy(s->to_dst_file); qemu_mutex_unlock_iothread(); =20 + /* Shutdown the postcopy fast path thread */ + if (migrate_postcopy_preempt()) { + postcopy_preempt_shutdown_file(s); + } + trace_migration_completion_postcopy_end_after_complete(); } else { goto fail; @@ -4125,6 +4162,15 @@ void migrate_fd_connect(MigrationState *s, Error *er= ror_in) } } =20 + /* This needs to be done before resuming a postcopy */ + if (postcopy_preempt_setup(s, &local_err)) { + error_report_err(local_err); + migrate_set_state(&s->state, MIGRATION_STATUS_SETUP, + MIGRATION_STATUS_FAILED); + migrate_fd_cleanup(s); + return; + } + if (resume) { /* Wakeup the main migration thread to do the recovery */ migrate_set_state(&s->state, MIGRATION_STATUS_POSTCOPY_PAUSED, diff --git a/migration/migration.h b/migration/migration.h index af4bcb19c2..caa910d956 100644 --- a/migration/migration.h +++ b/migration/migration.h @@ -23,6 +23,7 @@ #include "io/channel-buffer.h" #include "net/announce.h" #include "qom/object.h" +#include "postcopy-ram.h" =20 struct PostcopyBlocktimeContext; =20 @@ -112,6 +113,11 @@ struct MigrationIncomingState { * enabled. */ unsigned int postcopy_channels; + /* QEMUFile for postcopy only; it'll be handled by a separate thread */ + QEMUFile *postcopy_qemufile_dst; + /* Postcopy priority thread is used to receive postcopy requested page= s */ + QemuThread postcopy_prio_thread; + bool postcopy_prio_thread_created; /* * An array of temp host huge pages to be used, one for each postcopy * channel. @@ -192,6 +198,8 @@ struct MigrationState { QEMUBH *cleanup_bh; /* Protected by qemu_file_lock */ QEMUFile *to_dst_file; + /* Postcopy specific transfer channel */ + QEMUFile *postcopy_qemufile_src; QIOChannelBuffer *bioc; /* * Protects to_dst_file/from_dst_file pointers. We need to make sure = we diff --git a/migration/postcopy-ram.c b/migration/postcopy-ram.c index 32c52f4b1d..df0c02f729 100644 --- a/migration/postcopy-ram.c +++ b/migration/postcopy-ram.c @@ -33,6 +33,9 @@ #include "trace.h" #include "hw/boards.h" #include "exec/ramblock.h" +#include "socket.h" +#include "qemu-file-channel.h" +#include "yank_functions.h" =20 /* Arbitrary limit on size of each discard command, * keeps them around ~200 bytes @@ -567,6 +570,11 @@ int postcopy_ram_incoming_cleanup(MigrationIncomingSta= te *mis) { trace_postcopy_ram_incoming_cleanup_entry(); =20 + if (mis->postcopy_prio_thread_created) { + qemu_thread_join(&mis->postcopy_prio_thread); + mis->postcopy_prio_thread_created =3D false; + } + if (mis->have_fault_thread) { Error *local_err =3D NULL; =20 @@ -1102,8 +1110,13 @@ static int postcopy_temp_pages_setup(MigrationIncomi= ngState *mis) int err, i, channels; void *temp_page; =20 - /* TODO: will be boosted when enable postcopy preemption */ - mis->postcopy_channels =3D 1; + if (migrate_postcopy_preempt()) { + /* If preemption enabled, need extra channel for urgent requests */ + mis->postcopy_channels =3D RAM_CHANNEL_MAX; + } else { + /* Both precopy/postcopy on the same channel */ + mis->postcopy_channels =3D 1; + } =20 channels =3D mis->postcopy_channels; mis->postcopy_tmp_pages =3D g_malloc0_n(sizeof(PostcopyTmpPage), chann= els); @@ -1170,7 +1183,7 @@ int postcopy_ram_incoming_setup(MigrationIncomingStat= e *mis) return -1; } =20 - postcopy_thread_create(mis, &mis->fault_thread, "postcopy/fault", + postcopy_thread_create(mis, &mis->fault_thread, "fault-default", postcopy_ram_fault_thread, QEMU_THREAD_JOINABLE= ); mis->have_fault_thread =3D true; =20 @@ -1185,6 +1198,16 @@ int postcopy_ram_incoming_setup(MigrationIncomingSta= te *mis) return -1; } =20 + if (migrate_postcopy_preempt()) { + /* + * This thread needs to be created after the temp pages because it= 'll fetch + * RAM_CHANNEL_POSTCOPY PostcopyTmpPage immediately. + */ + postcopy_thread_create(mis, &mis->postcopy_prio_thread, "fault-fas= t", + postcopy_preempt_thread, QEMU_THREAD_JOINAB= LE); + mis->postcopy_prio_thread_created =3D true; + } + trace_postcopy_ram_enable_notify(); =20 return 0; @@ -1514,3 +1537,66 @@ void postcopy_unregister_shared_ufd(struct PostCopyF= D *pcfd) } } } + +bool postcopy_preempt_new_channel(MigrationIncomingState *mis, QEMUFile *f= ile) +{ + /* + * The new loading channel has its own threads, so it needs to be + * blocked too. It's by default true, just be explicit. + */ + qemu_file_set_blocking(file, true); + mis->postcopy_qemufile_dst =3D file; + trace_postcopy_preempt_new_channel(); + + /* Start the migration immediately */ + return true; +} + +int postcopy_preempt_setup(MigrationState *s, Error **errp) +{ + QIOChannel *ioc; + + if (!migrate_postcopy_preempt()) { + return 0; + } + + if (!migrate_multi_channels_is_allowed()) { + error_setg(errp, "Postcopy preempt is not supported as current " + "migration stream does not support multi-channels."); + return -1; + } + + ioc =3D socket_send_channel_create_sync(errp); + + if (ioc =3D=3D NULL) { + return -1; + } + + migration_ioc_register_yank(ioc); + s->postcopy_qemufile_src =3D qemu_fopen_channel_output(ioc); + + trace_postcopy_preempt_new_channel(); + + return 0; +} + +void *postcopy_preempt_thread(void *opaque) +{ + MigrationIncomingState *mis =3D opaque; + int ret; + + trace_postcopy_preempt_thread_entry(); + + rcu_register_thread(); + + qemu_sem_post(&mis->thread_sync_sem); + + /* Sending RAM_SAVE_FLAG_EOS to terminate this thread */ + ret =3D ram_load_postcopy(mis->postcopy_qemufile_dst, RAM_CHANNEL_POST= COPY); + + rcu_unregister_thread(); + + trace_postcopy_preempt_thread_exit(); + + return ret =3D=3D 0 ? NULL : (void *)-1; +} diff --git a/migration/postcopy-ram.h b/migration/postcopy-ram.h index 07684c0e1d..34b1080cde 100644 --- a/migration/postcopy-ram.h +++ b/migration/postcopy-ram.h @@ -183,4 +183,14 @@ int postcopy_wake_shared(struct PostCopyFD *pcfd, uint= 64_t client_addr, int postcopy_request_shared_page(struct PostCopyFD *pcfd, RAMBlock *rb, uint64_t client_addr, uint64_t offset); =20 +/* Hard-code channels for now for postcopy preemption */ +enum PostcopyChannels { + RAM_CHANNEL_PRECOPY =3D 0, + RAM_CHANNEL_POSTCOPY =3D 1, + RAM_CHANNEL_MAX, +}; + +bool postcopy_preempt_new_channel(MigrationIncomingState *mis, QEMUFile *f= ile); +int postcopy_preempt_setup(MigrationState *s, Error **errp); + #endif diff --git a/migration/ram.c b/migration/ram.c index 253fe4b756..c7ea1d9215 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -3644,15 +3644,15 @@ int ram_postcopy_incoming_init(MigrationIncomingSta= te *mis) * rcu_read_lock is taken prior to this being called. * * @f: QEMUFile where to send the data + * @channel: the channel to use for loading */ -int ram_load_postcopy(QEMUFile *f) +int ram_load_postcopy(QEMUFile *f, int channel) { int flags =3D 0, ret =3D 0; bool place_needed =3D false; bool matches_target_page_size =3D false; MigrationIncomingState *mis =3D migration_incoming_get_current(); - /* Currently we only use channel 0. TODO: use all the channels */ - PostcopyTmpPage *tmp_page =3D &mis->postcopy_tmp_pages[0]; + PostcopyTmpPage *tmp_page =3D &mis->postcopy_tmp_pages[channel]; =20 while (!ret && !(flags & RAM_SAVE_FLAG_EOS)) { ram_addr_t addr; @@ -3676,7 +3676,7 @@ int ram_load_postcopy(QEMUFile *f) flags =3D addr & ~TARGET_PAGE_MASK; addr &=3D TARGET_PAGE_MASK; =20 - trace_ram_load_postcopy_loop((uint64_t)addr, flags); + trace_ram_load_postcopy_loop(channel, (uint64_t)addr, flags); if (flags & (RAM_SAVE_FLAG_ZERO | RAM_SAVE_FLAG_PAGE | RAM_SAVE_FLAG_COMPRESS_PAGE)) { block =3D ram_block_from_stream(mis, f, flags); @@ -3717,10 +3717,10 @@ int ram_load_postcopy(QEMUFile *f) } else if (tmp_page->host_addr !=3D host_page_from_ram_block_offset(block, addr)) { /* not the 1st TP within the HP */ - error_report("Non-same host page detected. " + error_report("Non-same host page detected on channel %d: " "Target host page %p, received host page %p " "(rb %s offset 0x"RAM_ADDR_FMT" target_pages = %d)", - tmp_page->host_addr, + channel, tmp_page->host_addr, host_page_from_ram_block_offset(block, addr), block->idstr, addr, tmp_page->target_pages); ret =3D -EINVAL; @@ -4107,7 +4107,12 @@ static int ram_load(QEMUFile *f, void *opaque, int v= ersion_id) */ WITH_RCU_READ_LOCK_GUARD() { if (postcopy_running) { - ret =3D ram_load_postcopy(f); + /* + * Note! Here RAM_CHANNEL_PRECOPY is the precopy channel of + * postcopy migration, we have another RAM_CHANNEL_POSTCOPY to + * service fast page faults. + */ + ret =3D ram_load_postcopy(f, RAM_CHANNEL_PRECOPY); } else { ret =3D ram_load_precopy(f); } @@ -4269,6 +4274,12 @@ static int ram_resume_prepare(MigrationState *s, voi= d *opaque) return 0; } =20 +void postcopy_preempt_shutdown_file(MigrationState *s) +{ + qemu_put_be64(s->postcopy_qemufile_src, RAM_SAVE_FLAG_EOS); + qemu_fflush(s->postcopy_qemufile_src); +} + static SaveVMHandlers savevm_ram_handlers =3D { .save_setup =3D ram_save_setup, .save_live_iterate =3D ram_save_iterate, diff --git a/migration/ram.h b/migration/ram.h index ded0a3a086..5d90945a6e 100644 --- a/migration/ram.h +++ b/migration/ram.h @@ -61,7 +61,7 @@ void ram_postcopy_send_discard_bitmap(MigrationState *ms); /* For incoming postcopy discard */ int ram_discard_range(const char *block_name, uint64_t start, size_t lengt= h); int ram_postcopy_incoming_init(MigrationIncomingState *mis); -int ram_load_postcopy(QEMUFile *f); +int ram_load_postcopy(QEMUFile *f, int channel); =20 void ram_handle_compressed(void *host, uint8_t ch, uint64_t size); =20 @@ -73,6 +73,8 @@ int64_t ramblock_recv_bitmap_send(QEMUFile *file, const char *block_name); int ram_dirty_bitmap_reload(MigrationState *s, RAMBlock *rb); bool ramblock_page_is_discarded(RAMBlock *rb, ram_addr_t start); +void postcopy_preempt_shutdown_file(MigrationState *s); +void *postcopy_preempt_thread(void *opaque); =20 /* ram cache */ int colo_init_ram_cache(void); diff --git a/migration/savevm.c b/migration/savevm.c index d9076897b8..ecee05e631 100644 --- a/migration/savevm.c +++ b/migration/savevm.c @@ -2575,16 +2575,6 @@ static bool postcopy_pause_incoming(MigrationIncomin= gState *mis) { int i; =20 - /* - * If network is interrupted, any temp page we received will be useless - * because we didn't mark them as "received" in receivedmap. After a - * proper recovery later (which will sync src dirty bitmap with receiv= edmap - * on dest) these cached small pages will be resent again. - */ - for (i =3D 0; i < mis->postcopy_channels; i++) { - postcopy_temp_page_reset(&mis->postcopy_tmp_pages[i]); - } - trace_postcopy_pause_incoming(); =20 assert(migrate_postcopy_ram()); @@ -2613,6 +2603,16 @@ static bool postcopy_pause_incoming(MigrationIncomin= gState *mis) /* Notify the fault thread for the invalidated file handle */ postcopy_fault_thread_notify(mis); =20 + /* + * If network is interrupted, any temp page we received will be useless + * because we didn't mark them as "received" in receivedmap. After a + * proper recovery later (which will sync src dirty bitmap with receiv= edmap + * on dest) these cached small pages will be resent again. + */ + for (i =3D 0; i < mis->postcopy_channels; i++) { + postcopy_temp_page_reset(&mis->postcopy_tmp_pages[i]); + } + error_report("Detected IO failure for postcopy. " "Migration paused."); =20 diff --git a/migration/socket.c b/migration/socket.c index 05705a32d8..a7f345b353 100644 --- a/migration/socket.c +++ b/migration/socket.c @@ -26,7 +26,7 @@ #include "io/channel-socket.h" #include "io/net-listener.h" #include "trace.h" - +#include "postcopy-ram.h" =20 struct SocketOutgoingArgs { SocketAddress *saddr; @@ -39,6 +39,24 @@ void socket_send_channel_create(QIOTaskFunc f, void *dat= a) f, data, NULL, NULL); } =20 +QIOChannel *socket_send_channel_create_sync(Error **errp) +{ + QIOChannelSocket *sioc =3D qio_channel_socket_new(); + + if (!outgoing_args.saddr) { + object_unref(OBJECT(sioc)); + error_setg(errp, "Initial sock address not set!"); + return NULL; + } + + if (qio_channel_socket_connect_sync(sioc, outgoing_args.saddr, errp) <= 0) { + object_unref(OBJECT(sioc)); + return NULL; + } + + return QIO_CHANNEL(sioc); +} + int socket_send_channel_destroy(QIOChannel *send) { /* Remove channel */ @@ -158,6 +176,8 @@ socket_start_incoming_migration_internal(SocketAddress = *saddr, =20 if (migrate_use_multifd()) { num =3D migrate_multifd_channels(); + } else if (migrate_postcopy_preempt()) { + num =3D RAM_CHANNEL_MAX; } =20 if (qio_net_listener_open_sync(listener, saddr, num, errp) < 0) { diff --git a/migration/socket.h b/migration/socket.h index 891dbccceb..dc54df4e6c 100644 --- a/migration/socket.h +++ b/migration/socket.h @@ -21,6 +21,7 @@ #include "io/task.h" =20 void socket_send_channel_create(QIOTaskFunc f, void *data); +QIOChannel *socket_send_channel_create_sync(Error **errp); int socket_send_channel_destroy(QIOChannel *send); =20 void socket_start_incoming_migration(const char *str, Error **errp); diff --git a/migration/trace-events b/migration/trace-events index 1aec580e92..1f932782d9 100644 --- a/migration/trace-events +++ b/migration/trace-events @@ -91,7 +91,7 @@ migration_bitmap_clear_dirty(char *str, uint64_t start, u= int64_t size, unsigned migration_throttle(void) "" ram_discard_range(const char *rbname, uint64_t start, size_t len) "%s: sta= rt: %" PRIx64 " %zx" ram_load_loop(const char *rbname, uint64_t addr, int flags, void *host) "%= s: addr: 0x%" PRIx64 " flags: 0x%x host: %p" -ram_load_postcopy_loop(uint64_t addr, int flags) "@%" PRIx64 " %x" +ram_load_postcopy_loop(int channel, uint64_t addr, int flags) "chan=3D%d a= ddr=3D%" PRIx64 " flags=3D%x" ram_postcopy_send_discard_bitmap(void) "" ram_save_page(const char *rbname, uint64_t offset, void *host) "%s: offset= : 0x%" PRIx64 " host: %p" ram_save_queue_pages(const char *rbname, size_t start, size_t len) "%s: st= art: 0x%zx len: 0x%zx" @@ -278,6 +278,9 @@ postcopy_request_shared_page(const char *sharer, const = char *rb, uint64_t rb_off postcopy_request_shared_page_present(const char *sharer, const char *rb, u= int64_t rb_offset) "%s already %s offset 0x%"PRIx64 postcopy_wake_shared(uint64_t client_addr, const char *rb) "at 0x%"PRIx64"= in %s" postcopy_page_req_del(void *addr, int count) "resolved page req %p total %= d" +postcopy_preempt_new_channel(void) "" +postcopy_preempt_thread_entry(void) "" +postcopy_preempt_thread_exit(void) "" =20 get_mem_fault_cpu_index(int cpu, uint32_t pid) "cpu: %d, pid: %u" =20 --=20 2.32.0