From nobody Sun Nov 16 09:22:51 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=suse.de ARC-Seal: i=1; a=rsa-sha256; t=1741371411; cv=none; d=zohomail.com; s=zohoarc; b=hmq373Db8YuOqUhh4hW5Zlt3/cPdkag+mjjL+s5Yo6qoh9PDGs9bHmgqBuL/3xWgBS/UHwyuHeSkIxqkSl3Hyo/XIfVy2EbU5tK17z6ZO63QwIJrcuYBeaFPzN9bTSS5+wtyydaXJPw0LMPZ7/08CM8S9OdbMf9FDOHYLONxSR0= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1741371411; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=sCRIoHALaSzJJSqDE4A39LXrEmHIJBJiF0Llh+bdjrE=; b=eYSOlsMg8qG3DvCvYK07pVCdUJcqkY4jLKRRApFMKBqbb8au8Ihkk/euAO8D/uTjZmYEd6ZPbsyWDv5WavxKkshaNY5lTMcuiBQmjYe2+nLgQ+6whx9zpwPB+GWnh8dOXVErAIndXmMX6Y6js64XRudWUDCowyEw1Br59CwDsZg= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1741371411558699.9152735211754; Fri, 7 Mar 2025 10:16:51 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1tqcEz-0000iQ-Sh; Fri, 07 Mar 2025 13:16:09 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tqcEx-0000hj-Oj for qemu-devel@nongnu.org; Fri, 07 Mar 2025 13:16:07 -0500 Received: from smtp-out1.suse.de ([195.135.223.130]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1tqcEv-0004NI-6g for qemu-devel@nongnu.org; Fri, 07 Mar 2025 13:16:07 -0500 Received: from imap1.dmz-prg2.suse.org (unknown [10.150.64.97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id AD7F52118D; Fri, 7 Mar 2025 18:15:57 +0000 (UTC) Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 5783313A22; Fri, 7 Mar 2025 18:15:56 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id mC12Bdw3y2dNcgAAD6G6ig (envelope-from ); Fri, 07 Mar 2025 18:15:56 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1741371357; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=sCRIoHALaSzJJSqDE4A39LXrEmHIJBJiF0Llh+bdjrE=; b=CfaLtKmW8R+ZgyV9v/vRAPo785iS3SOUtZ+2TwcHGE39L94+7Fbd9gNpHRscAGXJgObxUV 3eWFLvP5QsQSYlrKGCXHMOKkFbaZDLIRSSSnye+PVLBiBDbXhi6AQoc3Ewl6TNnuQ4sO4v AKetBLyzACZEr+ROamzdaFeTDUaU7/4= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1741371357; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=sCRIoHALaSzJJSqDE4A39LXrEmHIJBJiF0Llh+bdjrE=; b=MBT6YbnO69Bc5p4U1Cc6DfMSAap0+QFrU5C5jshnRd/1LdZd8UH35+VrOMZHauLsEaqP48 l4eLV7RuCVv59FBw== Authentication-Results: smtp-out1.suse.de; none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1741371357; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=sCRIoHALaSzJJSqDE4A39LXrEmHIJBJiF0Llh+bdjrE=; b=CfaLtKmW8R+ZgyV9v/vRAPo785iS3SOUtZ+2TwcHGE39L94+7Fbd9gNpHRscAGXJgObxUV 3eWFLvP5QsQSYlrKGCXHMOKkFbaZDLIRSSSnye+PVLBiBDbXhi6AQoc3Ewl6TNnuQ4sO4v AKetBLyzACZEr+ROamzdaFeTDUaU7/4= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1741371357; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=sCRIoHALaSzJJSqDE4A39LXrEmHIJBJiF0Llh+bdjrE=; b=MBT6YbnO69Bc5p4U1Cc6DfMSAap0+QFrU5C5jshnRd/1LdZd8UH35+VrOMZHauLsEaqP48 l4eLV7RuCVv59FBw== From: Fabiano Rosas To: qemu-devel@nongnu.org Cc: Peter Xu , Yan Fu Subject: [PULL 1/8] migration: Fix UAF for incoming migration on MigrationState Date: Fri, 7 Mar 2025 15:15:44 -0300 Message-Id: <20250307181551.19887-2-farosas@suse.de> X-Mailer: git-send-email 2.35.3 In-Reply-To: <20250307181551.19887-1-farosas@suse.de> References: <20250307181551.19887-1-farosas@suse.de> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Spamd-Result: default: False [-2.80 / 50.00]; BAYES_HAM(-3.00)[100.00%]; MID_CONTAINS_FROM(1.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; R_MISSING_CHARSET(0.50)[]; NEURAL_HAM_SHORT(-0.20)[-1.000]; MIME_GOOD(-0.10)[text/plain]; MIME_TRACE(0.00)[0:+]; TO_DN_SOME(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; ARC_NA(0.00)[]; DKIM_SIGNED(0.00)[suse.de:s=susede2_rsa,suse.de:s=susede2_ed25519]; FUZZY_BLOCKED(0.00)[rspamd.com]; FROM_EQ_ENVFROM(0.00)[]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; RCVD_COUNT_TWO(0.00)[2]; TO_MATCH_ENVRCPT_ALL(0.00)[]; DBL_BLOCKED_OPENRESOLVER(0.00)[imap1.dmz-prg2.suse.org:helo]; RCVD_TLS_ALL(0.00)[] X-Spam-Score: -2.80 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=195.135.223.130; envelope-from=farosas@suse.de; helo=smtp-out1.suse.de X-Spam_score_int: -43 X-Spam_score: -4.4 X-Spam_bar: ---- X-Spam_report: (-4.4 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, RCVD_IN_VALIDITY_SAFE_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @suse.de) X-ZM-MESSAGEID: 1741371415017019100 Content-Type: text/plain; charset="utf-8" From: Peter Xu On the incoming migration side, QEMU uses a coroutine to load all the VM states. Inside, it may reference MigrationState on global states like migration capabilities, parameters, error state, shared mutexes and more. However there's nothing yet to make sure MigrationState won't get destroyed (e.g. after migration_shutdown()). Meanwhile there's also no API available to remove the incoming coroutine in migration_shutdown(), avoiding it to access the freed elements. There's a bug report showing this can happen and crash dest QEMU when migration is cancelled on source. When it happens, the dest main thread is trying to cleanup everything: #0 qemu_aio_coroutine_enter #1 aio_dispatch_handler #2 aio_poll #3 monitor_cleanup #4 qemu_cleanup #5 qemu_default_main Then it found the migration incoming coroutine, schedule it (even after migration_shutdown()), causing crash: #0 __pthread_kill_implementation #1 __pthread_kill_internal #2 __GI_raise #3 __GI_abort #4 __assert_fail_base #5 __assert_fail #6 qemu_mutex_lock_impl #7 qemu_lockable_mutex_lock #8 qemu_lockable_lock #9 qemu_lockable_auto_lock #10 migrate_set_error #11 process_incoming_migration_co #12 coroutine_trampoline To fix it, take a refcount after an incoming setup is properly done when qmp_migrate_incoming() succeeded the 1st time. As it's during a QMP handler which needs BQL, it means the main loop is still alive (without going into cleanups, which also needs BQL). Releasing the refcount now only until the incoming migration coroutine finished or failed. Hence the refcount is valid for both (1) setup phase of incoming ports, mostly IO watches (e.g. qio_channel_add_watch_full()), and (2) the incoming coroutine itself (process_incoming_migration_co()). Note that we can't unref in migration_incoming_state_destroy(), because both qmp_xen_load_devices_state() and load_snapshot() will use it without an incoming migration. Those hold BQL so they're not prone to this issue. PS: I suspect nobody uses Xen's command at all, as it didn't register yank, hence AFAIU the command should crash on master when trying to unregister yank in migration_incoming_state_destroy().. but that's another story. Also note that in some incoming failure cases we may not always unref the MigrationState refcount, which is a trade-off to keep things simple. We could make it accurate, but it can be an overkill. Some examples: - Unlike most of the rest protocols, socket_start_incoming_migration() may create net listener after incoming port setup sucessfully. It means we can't unref in migration_channel_process_incoming() as a generic path because socket protocol might keep using MigrationState. - For either socket or file, multiple IO watches might be created, it means logically each IO watch needs to take one refcount for MigrationState so as to be 100% accurate on ownership of refcount taken. In general, we at least need per-protocol handling to make it accurate, which can be an overkill if we know incoming failed after all. Add a short comment to explain that when taking the refcount in qmp_migrate_incoming(). Bugzilla: https://issues.redhat.com/browse/RHEL-69775 Tested-by: Yan Fu Signed-off-by: Peter Xu Reviewed-by: Fabiano Rosas Message-ID: <20250220132459.512610-1-peterx@redhat.com> Signed-off-by: Fabiano Rosas --- migration/migration.c | 40 ++++++++++++++++++++++++++++++++++++++-- 1 file changed, 38 insertions(+), 2 deletions(-) diff --git a/migration/migration.c b/migration/migration.c index 1833cfe358..d46e776e24 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -116,6 +116,27 @@ static void migration_downtime_start(MigrationState *s) s->downtime_start =3D qemu_clock_get_ms(QEMU_CLOCK_REALTIME); } =20 +/* + * This is unfortunate: incoming migration actually needs the outgoing + * migration state (MigrationState) to be there too, e.g. to query + * capabilities, parameters, using locks, setup errors, etc. + * + * NOTE: when calling this, making sure current_migration exists and not + * been freed yet! Otherwise trying to access the refcount is already + * an use-after-free itself.. + * + * TODO: Move shared part of incoming / outgoing out into separate object. + * Then this is not needed. + */ +static void migrate_incoming_ref_outgoing_state(void) +{ + object_ref(migrate_get_current()); +} +static void migrate_incoming_unref_outgoing_state(void) +{ + object_unref(migrate_get_current()); +} + static void migration_downtime_end(MigrationState *s) { int64_t now =3D qemu_clock_get_ms(QEMU_CLOCK_REALTIME); @@ -863,7 +884,7 @@ process_incoming_migration_co(void *opaque) * postcopy thread. */ trace_process_incoming_migration_co_postcopy_end_main(); - return; + goto out; } /* Else if something went wrong then just fall out of the normal e= xit */ } @@ -879,7 +900,8 @@ process_incoming_migration_co(void *opaque) } =20 migration_bh_schedule(process_incoming_migration_bh, mis); - return; + goto out; + fail: migrate_set_state(&mis->state, MIGRATION_STATUS_ACTIVE, MIGRATION_STATUS_FAILED); @@ -896,6 +918,9 @@ fail: =20 exit(EXIT_FAILURE); } +out: + /* Pairs with the refcount taken in qmp_migrate_incoming() */ + migrate_incoming_unref_outgoing_state(); } =20 /** @@ -1901,6 +1926,17 @@ void qmp_migrate_incoming(const char *uri, bool has_= channels, return; } =20 + /* + * Making sure MigrationState is available until incoming migration + * completes. + * + * NOTE: QEMU _might_ leak this refcount in some failure paths, but + * that's OK. This is the minimum change we need to at least making + * sure success case is clean on the refcount. We can try harder to + * make it accurate for any kind of failures, but it might be an + * overkill and doesn't bring us much benefit. + */ + migrate_incoming_ref_outgoing_state(); once =3D false; } =20 --=20 2.35.3 From nobody Sun Nov 16 09:22:51 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=suse.de ARC-Seal: i=1; a=rsa-sha256; t=1741371503; cv=none; d=zohomail.com; s=zohoarc; b=WwJL55OBq/hCpO90ZNpH6QAZ+GlCw1Y1pqGEEicPGXjQVsNORi4Ysd/kHHZYVwYScyQ2ojtlPM3/Qqjv52ilW3PirAjXq5v0l2X1IVoXiGHxHVI8ancZrmSevd2HCD0aKyVerCXXFMp2tXNZ+Bdd7mBtt0yKq+oaCjnU+RSxOCU= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1741371503; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=1i7apSn15jQatTnXZMO4odm/liOMWciUMfs5vbKW9dY=; b=S4yyXhNFrUlWZ7TFNOfqGbf1Ax7DfzPMfEqEnu/8B+jlmdJq3pxlwjeEXBaWOpBbv5pDv6Vv18rhZ7m4NLlMW0Cn//cSTgEyqUGZWcy7RQ3r2PyCxCvXxXvl2AT/6xlIpuR4ltzyElycwzvSDvi0qQWWKk8A/EdhtkEYq+M3kgM= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1741371503308312.7176149863411; Fri, 7 Mar 2025 10:18:23 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1tqcF6-0000j9-3w; Fri, 07 Mar 2025 13:16:16 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tqcF4-0000iw-NC for qemu-devel@nongnu.org; Fri, 07 Mar 2025 13:16:14 -0500 Received: from smtp-out1.suse.de ([2a07:de40:b251:101:10:150:64:1]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1tqcF2-0004eQ-9X for qemu-devel@nongnu.org; Fri, 07 Mar 2025 13:16:14 -0500 Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org [IPv6:2a07:de40:b281:104:10:150:64:97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id D7A9A21190; Fri, 7 Mar 2025 18:15:59 +0000 (UTC) Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 359E513A22; Fri, 7 Mar 2025 18:15:57 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id qNMOOd03y2dNcgAAD6G6ig (envelope-from ); Fri, 07 Mar 2025 18:15:57 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1741371360; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=1i7apSn15jQatTnXZMO4odm/liOMWciUMfs5vbKW9dY=; b=RuQ4stkppMwavXpTMBnvtB5+b0qZW80xdYeleRh1OqaH6TTonJTHPOCRy/EnruEZHNSnT/ HQPoKYq9ZHMZnD94n1fFSibIG14KeX7HuiHb35lmmpolZaPCMYd32hETdrcOJedLelyVIM eJefTtzJEVy86SLXWzD/X2vr8Uu7WM0= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1741371360; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=1i7apSn15jQatTnXZMO4odm/liOMWciUMfs5vbKW9dY=; b=VLyRGyoS0zGkuyi8gMxSXpbUQ6VJv6UlNOZX0GtILB+1dJXhV9R/DCQTcv6l1ca7KnslGh qCsXKD+UpgqpiLCA== Authentication-Results: smtp-out1.suse.de; dkim=pass header.d=suse.de header.s=susede2_rsa header.b=ePiWN5bn; dkim=pass header.d=suse.de header.s=susede2_ed25519 header.b=BiFB5SZG DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1741371359; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=1i7apSn15jQatTnXZMO4odm/liOMWciUMfs5vbKW9dY=; b=ePiWN5bnkK/Uguev+iO2d/FL9AWXx00iTSk8ydLhmSWKwoW1L3G73JRTM9oAkh/uYmi6qi DB270YjOaKjHhYBjs2kH0/EyVw6jXlwQ9xdkt8x6vV+mGOn0ZA5hcy+6OYg6PzLNA1yybq QxcanPUk6TgQ1i01b9t5sshhJOMyteM= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1741371359; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=1i7apSn15jQatTnXZMO4odm/liOMWciUMfs5vbKW9dY=; b=BiFB5SZGuBt+aEjkbZVpx2q0tt3sh5WsdkB9Y2I/jQIT+Jxzy7tW8woQ0DYS47IW1HXVyH 8yixcjQMT02uRPAQ== From: Fabiano Rosas To: qemu-devel@nongnu.org Cc: Peter Xu , Steve Sistare , David Hildenbrand Subject: [PULL 2/8] migration: ram block cpr blockers Date: Fri, 7 Mar 2025 15:15:45 -0300 Message-Id: <20250307181551.19887-3-farosas@suse.de> X-Mailer: git-send-email 2.35.3 In-Reply-To: <20250307181551.19887-1-farosas@suse.de> References: <20250307181551.19887-1-farosas@suse.de> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Rspamd-Queue-Id: D7A9A21190 X-Spam-Score: -3.01 X-Rspamd-Action: no action X-Spamd-Result: default: False [-3.01 / 50.00]; BAYES_HAM(-3.00)[100.00%]; NEURAL_HAM_LONG(-1.00)[-1.000]; MID_CONTAINS_FROM(1.00)[]; R_MISSING_CHARSET(0.50)[]; R_DKIM_ALLOW(-0.20)[suse.de:s=susede2_rsa,suse.de:s=susede2_ed25519]; NEURAL_HAM_SHORT(-0.20)[-1.000]; MIME_GOOD(-0.10)[text/plain]; MX_GOOD(-0.01)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; ARC_NA(0.00)[]; RCVD_COUNT_TWO(0.00)[2]; MIME_TRACE(0.00)[0:+]; RBL_SPAMHAUS_BLOCKED_OPENRESOLVER(0.00)[2a07:de40:b281:104:10:150:64:97:from]; SPAMHAUS_XBL(0.00)[2a07:de40:b281:104:10:150:64:97:from]; TO_DN_SOME(0.00)[]; RECEIVED_SPAMHAUS_BLOCKED_OPENRESOLVER(0.00)[2a07:de40:b281:106:10:150:64:167:received]; FUZZY_BLOCKED(0.00)[rspamd.com]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; DBL_BLOCKED_OPENRESOLVER(0.00)[imap1.dmz-prg2.suse.org:rdns,imap1.dmz-prg2.suse.org:helo,oracle.com:email]; RCVD_TLS_ALL(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; RCPT_COUNT_THREE(0.00)[4]; DKIM_SIGNED(0.00)[suse.de:s=susede2_rsa,suse.de:s=susede2_ed25519]; DKIM_TRACE(0.00)[suse.de:+] X-Rspamd-Server: rspamd1.dmz-prg2.suse.org Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=2a07:de40:b251:101:10:150:64:1; envelope-from=farosas@suse.de; helo=smtp-out1.suse.de X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @suse.de) (identity @suse.de) X-ZM-MESSAGEID: 1741371504977019000 Content-Type: text/plain; charset="utf-8" From: Steve Sistare Unlike cpr-reboot mode, cpr-transfer mode cannot save volatile ram blocks in the migration stream file and recreate them later, because the physical memory for the blocks is pinned and registered for vfio. Add a blocker for volatile ram blocks. Also add a blocker for RAM_GUEST_MEMFD. Preserving guest_memfd may be sufficient for CPR, but it has not been tested yet. Signed-off-by: Steve Sistare Reviewed-by: Fabiano Rosas Reviewed-by: Peter Xu Reviewed-by: David Hildenbrand Message-ID: <1740667681-257312-1-git-send-email-steven.sistare@oracle.com> Signed-off-by: Fabiano Rosas --- include/exec/memory.h | 3 ++ include/exec/ramblock.h | 1 + migration/savevm.c | 2 ++ system/physmem.c | 66 +++++++++++++++++++++++++++++++++++++++++ 4 files changed, 72 insertions(+) diff --git a/include/exec/memory.h b/include/exec/memory.h index 78c4e0aec8..d09af58c97 100644 --- a/include/exec/memory.h +++ b/include/exec/memory.h @@ -3203,6 +3203,9 @@ bool ram_block_discard_is_disabled(void); */ bool ram_block_discard_is_required(void); =20 +void ram_block_add_cpr_blocker(RAMBlock *rb, Error **errp); +void ram_block_del_cpr_blocker(RAMBlock *rb); + #endif =20 #endif diff --git a/include/exec/ramblock.h b/include/exec/ramblock.h index 0babd105c0..64484cd821 100644 --- a/include/exec/ramblock.h +++ b/include/exec/ramblock.h @@ -39,6 +39,7 @@ struct RAMBlock { /* RCU-enabled, writes protected by the ramlist lock */ QLIST_ENTRY(RAMBlock) next; QLIST_HEAD(, RAMBlockNotifier) ramblock_notifiers; + Error *cpr_blocker; int fd; uint64_t fd_offset; int guest_memfd; diff --git a/migration/savevm.c b/migration/savevm.c index 5c4fdfd95e..ce158c3512 100644 --- a/migration/savevm.c +++ b/migration/savevm.c @@ -3514,12 +3514,14 @@ void vmstate_register_ram(MemoryRegion *mr, DeviceS= tate *dev) qemu_ram_set_idstr(mr->ram_block, memory_region_name(mr), dev); qemu_ram_set_migratable(mr->ram_block); + ram_block_add_cpr_blocker(mr->ram_block, &error_fatal); } =20 void vmstate_unregister_ram(MemoryRegion *mr, DeviceState *dev) { qemu_ram_unset_idstr(mr->ram_block); qemu_ram_unset_migratable(mr->ram_block); + ram_block_del_cpr_blocker(mr->ram_block); } =20 void vmstate_register_ram_global(MemoryRegion *mr) diff --git a/system/physmem.c b/system/physmem.c index 8c1736f84e..445981a1b4 100644 --- a/system/physmem.c +++ b/system/physmem.c @@ -70,7 +70,10 @@ =20 #include "qemu/pmem.h" =20 +#include "qapi/qapi-types-migration.h" +#include "migration/blocker.h" #include "migration/cpr.h" +#include "migration/options.h" #include "migration/vmstate.h" =20 #include "qemu/range.h" @@ -1903,6 +1906,14 @@ static void ram_block_add(RAMBlock *new_block, Error= **errp) qemu_mutex_unlock_ramlist(); goto out_free; } + + error_setg(&new_block->cpr_blocker, + "Memory region %s uses guest_memfd, " + "which is not supported with CPR.", + memory_region_name(new_block->mr)); + migrate_add_blocker_modes(&new_block->cpr_blocker, errp, + MIG_MODE_CPR_TRANSFER, + -1); } =20 ram_size =3D (new_block->offset + new_block->max_length) >> TARGET_PAG= E_BITS; @@ -4094,3 +4105,58 @@ bool ram_block_discard_is_required(void) return qatomic_read(&ram_block_discard_required_cnt) || qatomic_read(&ram_block_coordinated_discard_required_cnt); } + +/* + * Return true if ram is compatible with CPR. Do not exclude rom, + * because the rom file could change in new QEMU. + */ +static bool ram_is_cpr_compatible(RAMBlock *rb) +{ + MemoryRegion *mr =3D rb->mr; + + if (!mr || !memory_region_is_ram(mr)) { + return true; + } + + /* Ram device is remapped in new QEMU */ + if (memory_region_is_ram_device(mr)) { + return true; + } + + /* + * A file descriptor is passed to new QEMU and remapped, or its backing + * file is reopened and mapped. It must be shared to avoid COW. + */ + if (rb->fd >=3D 0 && qemu_ram_is_shared(rb)) { + return true; + } + + return false; +} + +/* + * Add a blocker for each volatile ram block. This function should only be + * called after we know that the block is migratable. Non-migratable bloc= ks + * are either re-created in new QEMU, or are handled specially, or are cov= ered + * by a device-level CPR blocker. + */ +void ram_block_add_cpr_blocker(RAMBlock *rb, Error **errp) +{ + assert(qemu_ram_is_migratable(rb)); + + if (ram_is_cpr_compatible(rb)) { + return; + } + + error_setg(&rb->cpr_blocker, + "Memory region %s is not compatible with CPR. share=3Don is= " + "required for memory-backend objects, and aux-ram-share=3Do= n is " + "required.", memory_region_name(rb->mr)); + migrate_add_blocker_modes(&rb->cpr_blocker, errp, MIG_MODE_CPR_TRANSFE= R, + -1); +} + +void ram_block_del_cpr_blocker(RAMBlock *rb) +{ + migrate_del_blocker(&rb->cpr_blocker); +} --=20 2.35.3 From nobody Sun Nov 16 09:22:51 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=suse.de ARC-Seal: i=1; a=rsa-sha256; t=1741371511; cv=none; d=zohomail.com; s=zohoarc; b=AlEDMS1b1EMdSvRo5ohKcBDPeD02A9Qkk3X5rTEmKoh0v6gZWVHM3KfveULSooe41QHNH2U+R4cAoc2vaBNcpfSRKH/vcybQ/5nkVCT4OKLi016BoVDeNh6f7gjVf0JJLEStslv6efCOBidLaW4ARF5LhvjIhCRprirAm6nz/bU= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1741371511; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=D0wrgi6pxrkmOfIx5ZgU7+DOtdeSiALIzxKjJwUR95Y=; b=m2pf2QpVUJGs3v+xbTPqSFEF6DF9oYpSOs3rMMnqQ0rWSFpiuzcidN4Iae/UbMzn7kurZv/QNWOf6vYXx6KprOtiES7l+Eau7CUJBUilx3RrkMlirNVYfxsDf2y/qfQBzX1puI1L38+pt0LGAiaYADXpInYOyDzeMrt5Cyg5/xU= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 174137151138916.260445717200582; Fri, 7 Mar 2025 10:18:31 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1tqcEx-0000he-GW; Fri, 07 Mar 2025 13:16:07 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tqcEw-0000hW-2C for qemu-devel@nongnu.org; Fri, 07 Mar 2025 13:16:06 -0500 Received: from smtp-out2.suse.de ([195.135.223.131]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1tqcEu-0004Kk-DF for qemu-devel@nongnu.org; Fri, 07 Mar 2025 13:16:05 -0500 Received: from imap1.dmz-prg2.suse.org (unknown [10.150.64.97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id A670B1F395; Fri, 7 Mar 2025 18:16:01 +0000 (UTC) Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 52B1813A22; Fri, 7 Mar 2025 18:16:00 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id UM2NBOA3y2dNcgAAD6G6ig (envelope-from ); Fri, 07 Mar 2025 18:16:00 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1741371361; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=D0wrgi6pxrkmOfIx5ZgU7+DOtdeSiALIzxKjJwUR95Y=; b=1741Vt4ViN0IVcWerQoDOBKeHrWVjcTpZpd8WeTO+5IIKXuvZJ+vz4ODQl9HdAD6W1ph5Y z9FUkwUdLABgUOyIR+JLqWweE6nYeqJQRdlEVgR9Rj1voLjNbXGiXi3R/CbknA7LZJOzmK Zt742ZCSQKA4/mUNOyRiJ+/f9t2dVU8= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1741371361; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=D0wrgi6pxrkmOfIx5ZgU7+DOtdeSiALIzxKjJwUR95Y=; b=Jtf2kRDKJerhq+PzRgwQR6nc8f7Cog6J2uNpFxszFofLnpBNw3PmdsFbwzwNJWw/yestY8 mI9kKd8m6nGuEJCg== Authentication-Results: smtp-out2.suse.de; none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1741371361; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=D0wrgi6pxrkmOfIx5ZgU7+DOtdeSiALIzxKjJwUR95Y=; b=1741Vt4ViN0IVcWerQoDOBKeHrWVjcTpZpd8WeTO+5IIKXuvZJ+vz4ODQl9HdAD6W1ph5Y z9FUkwUdLABgUOyIR+JLqWweE6nYeqJQRdlEVgR9Rj1voLjNbXGiXi3R/CbknA7LZJOzmK Zt742ZCSQKA4/mUNOyRiJ+/f9t2dVU8= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1741371361; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=D0wrgi6pxrkmOfIx5ZgU7+DOtdeSiALIzxKjJwUR95Y=; b=Jtf2kRDKJerhq+PzRgwQR6nc8f7Cog6J2uNpFxszFofLnpBNw3PmdsFbwzwNJWw/yestY8 mI9kKd8m6nGuEJCg== From: Fabiano Rosas To: qemu-devel@nongnu.org Cc: Peter Xu , Li Zhijian Subject: [PULL 3/8] migration: Prioritize RDMA in ram_save_target_page() Date: Fri, 7 Mar 2025 15:15:46 -0300 Message-Id: <20250307181551.19887-4-farosas@suse.de> X-Mailer: git-send-email 2.35.3 In-Reply-To: <20250307181551.19887-1-farosas@suse.de> References: <20250307181551.19887-1-farosas@suse.de> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Spamd-Result: default: False [-2.80 / 50.00]; BAYES_HAM(-3.00)[100.00%]; MID_CONTAINS_FROM(1.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; R_MISSING_CHARSET(0.50)[]; NEURAL_HAM_SHORT(-0.20)[-0.999]; MIME_GOOD(-0.10)[text/plain]; MIME_TRACE(0.00)[0:+]; TO_DN_SOME(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; ARC_NA(0.00)[]; DKIM_SIGNED(0.00)[suse.de:s=susede2_rsa,suse.de:s=susede2_ed25519]; FUZZY_BLOCKED(0.00)[rspamd.com]; FROM_EQ_ENVFROM(0.00)[]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; RCVD_COUNT_TWO(0.00)[2]; TO_MATCH_ENVRCPT_ALL(0.00)[]; DBL_BLOCKED_OPENRESOLVER(0.00)[imap1.dmz-prg2.suse.org:helo]; RCVD_TLS_ALL(0.00)[] X-Spam-Score: -2.80 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=195.135.223.131; envelope-from=farosas@suse.de; helo=smtp-out2.suse.de X-Spam_score_int: -43 X-Spam_score: -4.4 X-Spam_bar: ---- X-Spam_report: (-4.4 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, RCVD_IN_VALIDITY_SAFE_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @suse.de) X-ZM-MESSAGEID: 1741371512670019000 Content-Type: text/plain; charset="utf-8" From: Li Zhijian Address an error in RDMA-based migration by ensuring RDMA is prioritized when saving pages in `ram_save_target_page()`. Previously, the RDMA protocol's page-saving step was placed after other protocols due to a refactoring in commit bc38dc2f5f3. This led to migration failures characterized by unknown control messages and state loading errors destination: (qemu) qemu-system-x86_64: Unknown control message QEMU FILE qemu-system-x86_64: error while loading state section id 1(ram) qemu-system-x86_64: load of migration failed: Operation not permitted source: (qemu) qemu-system-x86_64: RDMA is in an error state waiting migration to a= bort! qemu-system-x86_64: failed to save SaveStateEntry with id(name): 1(ram): -1 qemu-system-x86_64: rdma migration: recv polling control error! qemu-system-x86_64: warning: Early error. Sending error. qemu-system-x86_64: warning: rdma migration: send polling control error RDMA migration implemented its own protocol/method to send pages to destination side, hand over to RDMA first to prevent pages being saved by other protocol. Fixes: bc38dc2f5f3 ("migration: refactor ram_save_target_page functions") Reviewed-by: Peter Xu Signed-off-by: Li Zhijian Message-ID: <20250305062825.772629-2-lizhijian@fujitsu.com> Signed-off-by: Fabiano Rosas --- migration/ram.c | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-) diff --git a/migration/ram.c b/migration/ram.c index 589b6505eb..424df6d9f1 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -1964,6 +1964,11 @@ static int ram_save_target_page(RAMState *rs, PageSe= archStatus *pss) ram_addr_t offset =3D ((ram_addr_t)pss->page) << TARGET_PAGE_BITS; int res; =20 + /* Hand over to RDMA first */ + if (control_save_page(pss, offset, &res)) { + return res; + } + if (!migrate_multifd() || migrate_zero_page_detection() =3D=3D ZERO_PAGE_DETECTION_LEGACY= ) { if (save_zero_page(rs, pss, offset)) { @@ -1976,10 +1981,6 @@ static int ram_save_target_page(RAMState *rs, PageSe= archStatus *pss) return ram_save_multifd_page(block, offset); } =20 - if (control_save_page(pss, offset, &res)) { - return res; - } - return ram_save_page(rs, pss); } =20 --=20 2.35.3 From nobody Sun Nov 16 09:22:51 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=suse.de ARC-Seal: i=1; a=rsa-sha256; t=1741371498; cv=none; d=zohomail.com; s=zohoarc; b=hsf9wCwnMLsEmngqWLpA4fYh+H+lTxB1G3NJYUet4SUBcNLRJ5nAIiwUdp0mfMU0xGTF1IWYZfRXQvtuSEezVxUCZxS/kYBWjhfuFRHAcwnOdSMlODGOfhUuJ0phYt4GfqkaNpCknYOnk4qh8yfpTMi1+FGP058Mo6JF3mMSKg8= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1741371498; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=KCyu6U659hl4+BbhYS269+FXsnwSQqSXTjYAb5axr/s=; b=b5yETt/dPcRwzL73q2AWa5QIqGCq5MHP2l/WV6/gQcbxoJLVmrtjVAMSxM3ycU2sbKCIfzBcoqUQKIicWMiSS2+8A4eMg6BCAF/jt9mSGjG2XyZZL4D65fYXLhr+M53UCkNCcs842Mjex/ISI1cz/FlcgB9JeXvg/KooC61lFyA= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1741371498069802.6115318758898; Fri, 7 Mar 2025 10:18:18 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1tqcF6-0000jN-II; Fri, 07 Mar 2025 13:16:16 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tqcF4-0000io-3v for qemu-devel@nongnu.org; Fri, 07 Mar 2025 13:16:14 -0500 Received: from smtp-out2.suse.de ([2a07:de40:b251:101:10:150:64:2]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1tqcF1-0004Z1-Gw for qemu-devel@nongnu.org; Fri, 07 Mar 2025 13:16:13 -0500 Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org [IPv6:2a07:de40:b281:104:10:150:64:97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id 830371F441; Fri, 7 Mar 2025 18:16:03 +0000 (UTC) Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 2E59513A22; Fri, 7 Mar 2025 18:16:01 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id 2M9DN+E3y2dNcgAAD6G6ig (envelope-from ); Fri, 07 Mar 2025 18:16:01 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1741371364; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=KCyu6U659hl4+BbhYS269+FXsnwSQqSXTjYAb5axr/s=; b=AwPEwXEVPViUgxwRmDv+iFsFGeYL/0sxfXj1mlFeD5IpedAyPF452KnoHbpqqFGPZppZPT Dvo1QbiKX2rN7A04K6fzNYcIxC1r26VsgSXVy0ALaxMGs7mEzfCt2gEAQA7axReIelyiDt iR1VK1mxoAMcovYvnoX0xtqDwLi89Ww= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1741371364; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=KCyu6U659hl4+BbhYS269+FXsnwSQqSXTjYAb5axr/s=; b=KFVa/qgtUAQOlE5RX3g9YF1PcCKtKKLt6biiMK3YOkh8V45TS7DqRF3v1h7dliYi+DGhHD oFXGPc95Ux4QgxCQ== Authentication-Results: smtp-out2.suse.de; dkim=pass header.d=suse.de header.s=susede2_rsa header.b=qlzBqS9O; dkim=pass header.d=suse.de header.s=susede2_ed25519 header.b=Ya1DQomK DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1741371363; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=KCyu6U659hl4+BbhYS269+FXsnwSQqSXTjYAb5axr/s=; b=qlzBqS9OVE7vMsQR9IpiJDrAwZLJg866cGXWc6qqIAWnNpAvLiJ8PLzHO8VyDjpVMUvxSQ 0ma+xJuw7Ez32UaisQMDcLxFzZnMfjyiRisP5s4umwvG59bQuFseYYzm8DFY/RCxbB7hsI mJh3hKJYkMkFB2g6mC808G/wYoQvhTU= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1741371363; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=KCyu6U659hl4+BbhYS269+FXsnwSQqSXTjYAb5axr/s=; b=Ya1DQomKB0PQprrhVj8IUVoIPku062oURgpA+7KyIu3dpa2fYoTrrXb7jFXjGGxMEh4/xt gs1AdUXTy17lxrDg== From: Fabiano Rosas To: qemu-devel@nongnu.org Cc: Peter Xu , Li Zhijian Subject: [PULL 4/8] migration: check RDMA and capabilities are compatible on both sides Date: Fri, 7 Mar 2025 15:15:47 -0300 Message-Id: <20250307181551.19887-5-farosas@suse.de> X-Mailer: git-send-email 2.35.3 In-Reply-To: <20250307181551.19887-1-farosas@suse.de> References: <20250307181551.19887-1-farosas@suse.de> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Rspamd-Queue-Id: 830371F441 X-Spam-Score: -3.01 X-Rspamd-Action: no action X-Spamd-Result: default: False [-3.01 / 50.00]; BAYES_HAM(-3.00)[99.99%]; NEURAL_HAM_LONG(-1.00)[-1.000]; MID_CONTAINS_FROM(1.00)[]; R_MISSING_CHARSET(0.50)[]; R_DKIM_ALLOW(-0.20)[suse.de:s=susede2_rsa,suse.de:s=susede2_ed25519]; NEURAL_HAM_SHORT(-0.20)[-1.000]; MIME_GOOD(-0.10)[text/plain]; MX_GOOD(-0.01)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; ARC_NA(0.00)[]; RCVD_COUNT_TWO(0.00)[2]; MIME_TRACE(0.00)[0:+]; RBL_SPAMHAUS_BLOCKED_OPENRESOLVER(0.00)[2a07:de40:b281:104:10:150:64:97:from]; SPAMHAUS_XBL(0.00)[2a07:de40:b281:104:10:150:64:97:from]; TO_DN_SOME(0.00)[]; RECEIVED_SPAMHAUS_BLOCKED_OPENRESOLVER(0.00)[2a07:de40:b281:106:10:150:64:167:received]; FUZZY_BLOCKED(0.00)[rspamd.com]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; DBL_BLOCKED_OPENRESOLVER(0.00)[imap1.dmz-prg2.suse.org:rdns,imap1.dmz-prg2.suse.org:helo]; RCVD_TLS_ALL(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; DKIM_SIGNED(0.00)[suse.de:s=susede2_rsa,suse.de:s=susede2_ed25519]; DKIM_TRACE(0.00)[suse.de:+] X-Rspamd-Server: rspamd1.dmz-prg2.suse.org Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=2a07:de40:b251:101:10:150:64:2; envelope-from=farosas@suse.de; helo=smtp-out2.suse.de X-Spam_score_int: -43 X-Spam_score: -4.4 X-Spam_bar: ---- X-Spam_report: (-4.4 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @suse.de) (identity @suse.de) X-ZM-MESSAGEID: 1741371499702019100 Content-Type: text/plain; charset="utf-8" From: Li Zhijian Depending on the order of starting RDMA and setting capability, they can be categorized into the following scenarios: Source: S1: [set capabilities] -> [Start RDMA outgoing] Destination: D1: [set capabilities] -> [Start RDMA incoming] D2: [Start RDMA incoming] -> [set capabilities] Previously, compatibility between RDMA and capabilities was verified only in scenario D1, potentially causing migration failures in other situations. For scenarios S1 and D1, we can seamlessly incorporate migration_transport_compatible() to address compatibility between channels and capabilities vs transport. For scenario D2, ensure compatibility within migrate_caps_check(). Reviewed-by: Peter Xu Signed-off-by: Li Zhijian Message-ID: <20250305062825.772629-3-lizhijian@fujitsu.com> [fixup comment alignment] Signed-off-by: Fabiano Rosas --- migration/migration.c | 30 ++++++++++++++++++++---------- migration/options.c | 21 +++++++++++++++++++++ migration/options.h | 1 + 3 files changed, 42 insertions(+), 10 deletions(-) diff --git a/migration/migration.c b/migration/migration.c index d46e776e24..22dd966970 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -259,6 +259,24 @@ migration_channels_and_transport_compatible(MigrationA= ddress *addr, return true; } =20 +static bool +migration_capabilities_and_transport_compatible(MigrationAddress *addr, + Error **errp) +{ + if (addr->transport =3D=3D MIGRATION_ADDRESS_TYPE_RDMA) { + return migrate_rdma_caps_check(migrate_get_current()->capabilities, + errp); + } + + return true; +} + +static bool migration_transport_compatible(MigrationAddress *addr, Error *= *errp) +{ + return migration_channels_and_transport_compatible(addr, errp) && + migration_capabilities_and_transport_compatible(addr, errp); +} + static gint page_request_addr_cmp(gconstpointer ap, gconstpointer bp) { uintptr_t a =3D (uintptr_t) ap, b =3D (uintptr_t) bp; @@ -750,7 +768,7 @@ static void qemu_start_incoming_migration(const char *u= ri, bool has_channels, } =20 /* transport mechanism not suitable for migration? */ - if (!migration_channels_and_transport_compatible(addr, errp)) { + if (!migration_transport_compatible(addr, errp)) { return; } =20 @@ -769,14 +787,6 @@ static void qemu_start_incoming_migration(const char *= uri, bool has_channels, } #ifdef CONFIG_RDMA } else if (addr->transport =3D=3D MIGRATION_ADDRESS_TYPE_RDMA) { - if (migrate_xbzrle()) { - error_setg(errp, "RDMA and XBZRLE can't be used together"); - return; - } - if (migrate_multifd()) { - error_setg(errp, "RDMA and multifd can't be used together"); - return; - } rdma_start_incoming_migration(&addr->u.rdma, errp); #endif } else if (addr->transport =3D=3D MIGRATION_ADDRESS_TYPE_EXEC) { @@ -2208,7 +2218,7 @@ void qmp_migrate(const char *uri, bool has_channels, } =20 /* transport mechanism not suitable for migration? */ - if (!migration_channels_and_transport_compatible(addr, errp)) { + if (!migration_transport_compatible(addr, errp)) { return; } =20 diff --git a/migration/options.c b/migration/options.c index b0ac2ea408..1f3602839d 100644 --- a/migration/options.c +++ b/migration/options.c @@ -448,6 +448,20 @@ static bool migrate_incoming_started(void) return !!migration_incoming_get_current()->transport_data; } =20 +bool migrate_rdma_caps_check(bool *caps, Error **errp) +{ + if (caps[MIGRATION_CAPABILITY_XBZRLE]) { + error_setg(errp, "RDMA and XBZRLE can't be used together"); + return false; + } + if (caps[MIGRATION_CAPABILITY_MULTIFD]) { + error_setg(errp, "RDMA and multifd can't be used together"); + return false; + } + + return true; +} + /** * @migration_caps_check - check capability compatibility * @@ -611,6 +625,13 @@ bool migrate_caps_check(bool *old_caps, bool *new_caps= , Error **errp) } } =20 + /* + * On destination side, check the cases that capability is being set + * after incoming thread has started. + */ + if (migrate_rdma() && !migrate_rdma_caps_check(new_caps, errp)) { + return false; + } return true; } =20 diff --git a/migration/options.h b/migration/options.h index 762be4e641..82d839709e 100644 --- a/migration/options.h +++ b/migration/options.h @@ -57,6 +57,7 @@ bool migrate_tls(void); =20 /* capabilities helpers */ =20 +bool migrate_rdma_caps_check(bool *caps, Error **errp); bool migrate_caps_check(bool *old_caps, bool *new_caps, Error **errp); =20 /* parameters */ --=20 2.35.3 From nobody Sun Nov 16 09:22:51 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=suse.de ARC-Seal: i=1; a=rsa-sha256; t=1741371451; cv=none; d=zohomail.com; s=zohoarc; b=c9xZYQfUClwWM+SaByzwMPu6/F0xEXbRttxSsEv8u1C62HQGK8F1jvTJTXLJHjqjhEO7FVqfY8clVZu+ZI7Hcx218041p/uEIKyzUQKjDg3M+S9kWLqJUbzvPXQ/HTTZG008Wq+6gKEGPdKdNIykQWrvNahdyYfuebnRjFn4H6o= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1741371451; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=lpBbc8MkhNdLIdzHAxQIMGOUAibhSlDA9kfQwVxsW/g=; b=Ojhzhjx7qwI3ig5+JPwSSXobpfWe+KcP1Iq4EZJXgMnXaG2NNaFBfftp0GAakfZ10VP8vLtQ27irkK9JpTPp2SwG/5ZFz5A9Eeyn055bcOledBeUd3dY8nG5QfGvgdTmaI69EDRsFg9oc8nqWQT8CTZCE3ZJ2LWTdmtULmP0BHU= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1741371451390654.3164972516546; Fri, 7 Mar 2025 10:17:31 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1tqcFC-0000l2-FW; Fri, 07 Mar 2025 13:16:22 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tqcFA-0000k9-9t for qemu-devel@nongnu.org; Fri, 07 Mar 2025 13:16:20 -0500 Received: from smtp-out1.suse.de ([2a07:de40:b251:101:10:150:64:1]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1tqcF8-0004xB-MJ for qemu-devel@nongnu.org; Fri, 07 Mar 2025 13:16:20 -0500 Received: from imap1.dmz-prg2.suse.org (unknown [10.150.64.97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id 5F4C92118C; Fri, 7 Mar 2025 18:16:05 +0000 (UTC) Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 0B43C13A53; Fri, 7 Mar 2025 18:16:03 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id AMeqLuM3y2dNcgAAD6G6ig (envelope-from ); Fri, 07 Mar 2025 18:16:03 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1741371366; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=lpBbc8MkhNdLIdzHAxQIMGOUAibhSlDA9kfQwVxsW/g=; b=iVBjhwZThhD17e06Z2kDS6bInNECaApFdoEfXLRWhbTvL3+x83KPlkRzrY87gVCVqXzGvB LdV3USNvYRZXW8740z30c3fLZnKu0q40HVTcqOYeykguaHMvB0SSsJ9jQiR8k8i6uGSUCo lSRoBIdVuLNwU7QTJYJsqf1Zc0MOw6Q= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1741371366; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=lpBbc8MkhNdLIdzHAxQIMGOUAibhSlDA9kfQwVxsW/g=; b=6Puzy5cKzDBljALJ6OhHby9BQiyXkTodX2oZRv6XydGkXXCYMpDsUbIrKflZczGixbBKo2 W43ThpFGxgBIqkCQ== Authentication-Results: smtp-out1.suse.de; none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1741371365; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=lpBbc8MkhNdLIdzHAxQIMGOUAibhSlDA9kfQwVxsW/g=; b=s+t5DE1ifgAFO78y22t+hT8HgH8jByiclVXJhV6J5ZabAOZQhUPlJrZRqtngRYPBm0W0tY HEuELUUhY+tXa8772U/J1ROyWJyMKpoLJlaIJ4+kv+MHlikOFJ4279DTPEs8FSgg77nK/g 2eYMQ6jWinKnpBQ3ctT+LzrOibRvxgY= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1741371365; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=lpBbc8MkhNdLIdzHAxQIMGOUAibhSlDA9kfQwVxsW/g=; b=UacnNoViI6WMYIuyTjAC3UWqqVWt0MTFWVAUhSKWJc0S/9c2/VtJa/kx2VQ6Wx9Oq8Qo6b IFt1XeZK5W32caDA== From: Fabiano Rosas To: qemu-devel@nongnu.org Cc: Peter Xu , Li Zhijian Subject: [PULL 5/8] migration: disable RDMA + postcopy-ram Date: Fri, 7 Mar 2025 15:15:48 -0300 Message-Id: <20250307181551.19887-6-farosas@suse.de> X-Mailer: git-send-email 2.35.3 In-Reply-To: <20250307181551.19887-1-farosas@suse.de> References: <20250307181551.19887-1-farosas@suse.de> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Spamd-Result: default: False [-2.80 / 50.00]; BAYES_HAM(-3.00)[99.99%]; MID_CONTAINS_FROM(1.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; R_MISSING_CHARSET(0.50)[]; NEURAL_HAM_SHORT(-0.20)[-0.999]; MIME_GOOD(-0.10)[text/plain]; MIME_TRACE(0.00)[0:+]; TO_DN_SOME(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; ARC_NA(0.00)[]; DKIM_SIGNED(0.00)[suse.de:s=susede2_rsa,suse.de:s=susede2_ed25519]; FUZZY_BLOCKED(0.00)[rspamd.com]; FROM_EQ_ENVFROM(0.00)[]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; RCVD_COUNT_TWO(0.00)[2]; TO_MATCH_ENVRCPT_ALL(0.00)[]; DBL_BLOCKED_OPENRESOLVER(0.00)[suse.de:email,suse.de:mid,imap1.dmz-prg2.suse.org:helo]; RCVD_TLS_ALL(0.00)[] X-Spam-Score: -2.80 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=2a07:de40:b251:101:10:150:64:1; envelope-from=farosas@suse.de; helo=smtp-out1.suse.de X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @suse.de) (identity @suse.de) X-ZM-MESSAGEID: 1741371453717019100 Content-Type: text/plain; charset="utf-8" From: Li Zhijian It's believed that RDMA + postcopy-ram has been broken for a while. Rather than spending time re-enabling it, let's simply disable it as a trade-off. Reviewed-by: Peter Xu Signed-off-by: Li Zhijian Message-ID: <20250305062825.772629-4-lizhijian@fujitsu.com> Signed-off-by: Fabiano Rosas --- migration/options.c | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/migration/options.c b/migration/options.c index 1f3602839d..4ba8fcb7dc 100644 --- a/migration/options.c +++ b/migration/options.c @@ -458,6 +458,10 @@ bool migrate_rdma_caps_check(bool *caps, Error **errp) error_setg(errp, "RDMA and multifd can't be used together"); return false; } + if (caps[MIGRATION_CAPABILITY_POSTCOPY_RAM]) { + error_setg(errp, "RDMA and postcopy-ram can't be used together"); + return false; + } =20 return true; } --=20 2.35.3 From nobody Sun Nov 16 09:22:51 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=suse.de ARC-Seal: i=1; a=rsa-sha256; t=1741371497; cv=none; d=zohomail.com; s=zohoarc; b=XIaJHO8WYKdgkXsRr6puWLIBwF0tE3grv6Ro7Xd7jE9pK3NOcwi3RVtBQd7NPH1r9UCqHbdOdyKN8XrJVL1UM//isnE1UADi+k+DD9WT+eTrH4s5kegj4+GCUN/y3n+VO21Jfc99pl3oAoFysvf+unZ4U1tt4zrG3EBgkO6mOtE= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1741371497; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=TY/xtH07fdrTTFdSQHkFWSKdtEQrZw0frhISX/MFn0w=; b=HiJNexICTkBQ9NVsTKBI18ghG9ZA5+nZNXqb3cHj5Gs3th+gU5Sw+QrqqtSiH4vr0+o1mXdCv1eWmZezr+ZrHXo7M+8ToBO2vX0iVd6J5PnATueSZvBqTCPy0qCHpPQoisQlSYAOOaAv0GNItBN3GfZWk7g/7W7KmrNlCmX/5ZI= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1741371497450370.2251288296012; Fri, 7 Mar 2025 10:18:17 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1tqcFH-0000mp-5G; Fri, 07 Mar 2025 13:16:27 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tqcFF-0000mU-UK for qemu-devel@nongnu.org; Fri, 07 Mar 2025 13:16:26 -0500 Received: from smtp-out1.suse.de ([2a07:de40:b251:101:10:150:64:1]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1tqcFE-0005EK-3M for qemu-devel@nongnu.org; Fri, 07 Mar 2025 13:16:25 -0500 Received: from imap1.dmz-prg2.suse.org (unknown [10.150.64.97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id 3D02A21193; Fri, 7 Mar 2025 18:16:07 +0000 (UTC) Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id DB65F13A22; Fri, 7 Mar 2025 18:16:05 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id ODXbJeU3y2dNcgAAD6G6ig (envelope-from ); Fri, 07 Mar 2025 18:16:05 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1741371367; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=TY/xtH07fdrTTFdSQHkFWSKdtEQrZw0frhISX/MFn0w=; b=q6dsjEo6ESekgob2IslR0Gr3XpJgsK3D4Mr0npC9Lrz6xfwzu8g/MaBKCo7Oc9svut3m5U CXks5wBKzJe/DEOsyGdYrb+kU5NvKmHo2jUOSmflsK3mds8dVjqbxkHihc3IEzyI9B1Tk4 NdUZ88lqgOwh/oUUpvSd80D1PasE6wo= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1741371367; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=TY/xtH07fdrTTFdSQHkFWSKdtEQrZw0frhISX/MFn0w=; b=2fQiqp8bWF85i9rJkccvfDIpue16U2AqkATpv4slhfcwplFrNqOATfQVQpRqCXavj7YuSS bszneV8eWG0AkRDw== Authentication-Results: smtp-out1.suse.de; none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1741371367; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=TY/xtH07fdrTTFdSQHkFWSKdtEQrZw0frhISX/MFn0w=; b=q6dsjEo6ESekgob2IslR0Gr3XpJgsK3D4Mr0npC9Lrz6xfwzu8g/MaBKCo7Oc9svut3m5U CXks5wBKzJe/DEOsyGdYrb+kU5NvKmHo2jUOSmflsK3mds8dVjqbxkHihc3IEzyI9B1Tk4 NdUZ88lqgOwh/oUUpvSd80D1PasE6wo= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1741371367; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=TY/xtH07fdrTTFdSQHkFWSKdtEQrZw0frhISX/MFn0w=; b=2fQiqp8bWF85i9rJkccvfDIpue16U2AqkATpv4slhfcwplFrNqOATfQVQpRqCXavj7YuSS bszneV8eWG0AkRDw== From: Fabiano Rosas To: qemu-devel@nongnu.org Cc: Peter Xu , Li Zhijian Subject: [PULL 6/8] migration/rdma: Remove redundant migration_in_postcopy checks Date: Fri, 7 Mar 2025 15:15:49 -0300 Message-Id: <20250307181551.19887-7-farosas@suse.de> X-Mailer: git-send-email 2.35.3 In-Reply-To: <20250307181551.19887-1-farosas@suse.de> References: <20250307181551.19887-1-farosas@suse.de> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Spam-Score: -2.80 X-Spamd-Result: default: False [-2.80 / 50.00]; BAYES_HAM(-3.00)[99.99%]; MID_CONTAINS_FROM(1.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; R_MISSING_CHARSET(0.50)[]; NEURAL_HAM_SHORT(-0.20)[-0.999]; MIME_GOOD(-0.10)[text/plain]; MIME_TRACE(0.00)[0:+]; TO_DN_SOME(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; ARC_NA(0.00)[]; DKIM_SIGNED(0.00)[suse.de:s=susede2_rsa,suse.de:s=susede2_ed25519]; FUZZY_BLOCKED(0.00)[rspamd.com]; FROM_EQ_ENVFROM(0.00)[]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; RCVD_COUNT_TWO(0.00)[2]; TO_MATCH_ENVRCPT_ALL(0.00)[]; DBL_BLOCKED_OPENRESOLVER(0.00)[imap1.dmz-prg2.suse.org:helo,suse.de:mid,suse.de:email]; RCVD_TLS_ALL(0.00)[] Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=2a07:de40:b251:101:10:150:64:1; envelope-from=farosas@suse.de; helo=smtp-out1.suse.de X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @suse.de) X-ZM-MESSAGEID: 1741371499622019100 Content-Type: text/plain; charset="utf-8" From: Li Zhijian Since we have disabled RDMA + postcopy, it's safe to remove the migration_in_postcopy() that follows the migrate_rdma(). Reviewed-by: Peter Xu Signed-off-by: Li Zhijian Message-ID: <20250305062825.772629-5-lizhijian@fujitsu.com> Signed-off-by: Fabiano Rosas --- migration/rdma.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/migration/rdma.c b/migration/rdma.c index 76fb034923..e5b4ac599b 100644 --- a/migration/rdma.c +++ b/migration/rdma.c @@ -3284,7 +3284,7 @@ err: int rdma_control_save_page(QEMUFile *f, ram_addr_t block_offset, ram_addr_t offset, size_t size) { - if (!migrate_rdma() || migration_in_postcopy()) { + if (!migrate_rdma()) { return RAM_SAVE_CONTROL_NOT_SUPP; } =20 @@ -3829,7 +3829,7 @@ int rdma_block_notification_handle(QEMUFile *f, const= char *name) =20 int rdma_registration_start(QEMUFile *f, uint64_t flags) { - if (!migrate_rdma() || migration_in_postcopy()) { + if (!migrate_rdma()) { return 0; } =20 @@ -3861,7 +3861,7 @@ int rdma_registration_stop(QEMUFile *f, uint64_t flag= s) RDMAControlHeader head =3D { .len =3D 0, .repeat =3D 1 }; int ret; =20 - if (!migrate_rdma() || migration_in_postcopy()) { + if (!migrate_rdma()) { return 0; } =20 --=20 2.35.3 From nobody Sun Nov 16 09:22:51 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=suse.de ARC-Seal: i=1; a=rsa-sha256; t=1741371511; cv=none; d=zohomail.com; s=zohoarc; b=GzXZ/WmeWqf1Lo2rdYen+4/o5BU8oTgu8qIeOnwko9bKEQrIK8OMbHQj9zDEaJDOnWiemMXLpm3hwwIDYR327kh3tj7U5yfgpAOXJFTgTUOJZx6L7FBsBCzqej8TfmjeVu3HI4ErPjzoMv1PDO7EFOdZpISYmrWZBDso3FL2Igs= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1741371511; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=LrUs++PLGKZhWmWTEUi2shbFpx1DoqHEvexa17ejjMM=; b=jNDnMGsbWcMmhnmFiZFtzOKaFXHRttbOeKfa5EkfHuBQ5UsNzNNqkDcAzCMCWYnPOEMLK7hhuOzoI3BGgO6nuAoQ0n7LZIJSGfrFfYZyOaFjx3OSXzOq07KKOBOz7BsVKQg3eqxzy2W/F4UtlRP4C6k7Jlu8GbCdAKg7VHwhYhs= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1741371511817650.703986466396; Fri, 7 Mar 2025 10:18:31 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1tqcFO-0000pW-73; Fri, 07 Mar 2025 13:16:34 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tqcFL-0000oV-Tw for qemu-devel@nongnu.org; Fri, 07 Mar 2025 13:16:31 -0500 Received: from smtp-out1.suse.de ([195.135.223.130]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1tqcFJ-0005MU-Sw for qemu-devel@nongnu.org; Fri, 07 Mar 2025 13:16:31 -0500 Received: from imap1.dmz-prg2.suse.org (unknown [10.150.64.97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id 2521D2118D; Fri, 7 Mar 2025 18:16:09 +0000 (UTC) Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id C353F13A53; Fri, 7 Mar 2025 18:16:07 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id 6BL/Huc3y2dNcgAAD6G6ig (envelope-from ); Fri, 07 Mar 2025 18:16:07 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1741371369; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=LrUs++PLGKZhWmWTEUi2shbFpx1DoqHEvexa17ejjMM=; b=1Nvp/vjVIBTWlmMooKH/z2SxDn8tjsG81X7F+JkJEDOZ2HMiTjCAWFLOiY0U2GzDKhs4qz JlFO4V59yXynXYXPgmXzYWN+2Trf5e8/BselY3Ifi4Kqvm22QYvw8sR6StHeAG8JRNLyxj VWWNtcYcU2X0n6jS9lxnmHpMkOO4ZNA= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1741371369; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=LrUs++PLGKZhWmWTEUi2shbFpx1DoqHEvexa17ejjMM=; b=xsFeeudMT4Yg/FcjmNMX+AwGV1rC4gyd5kOgkiYR99hvU3QMrxPmuu620Y9yhMFWPjtW56 JfHPuGB5/KUBwACg== Authentication-Results: smtp-out1.suse.de; none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1741371369; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=LrUs++PLGKZhWmWTEUi2shbFpx1DoqHEvexa17ejjMM=; b=1Nvp/vjVIBTWlmMooKH/z2SxDn8tjsG81X7F+JkJEDOZ2HMiTjCAWFLOiY0U2GzDKhs4qz JlFO4V59yXynXYXPgmXzYWN+2Trf5e8/BselY3Ifi4Kqvm22QYvw8sR6StHeAG8JRNLyxj VWWNtcYcU2X0n6jS9lxnmHpMkOO4ZNA= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1741371369; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=LrUs++PLGKZhWmWTEUi2shbFpx1DoqHEvexa17ejjMM=; b=xsFeeudMT4Yg/FcjmNMX+AwGV1rC4gyd5kOgkiYR99hvU3QMrxPmuu620Y9yhMFWPjtW56 JfHPuGB5/KUBwACg== From: Fabiano Rosas To: qemu-devel@nongnu.org Cc: Peter Xu , Li Zhijian Subject: [PULL 7/8] migration: Unfold control_save_page() Date: Fri, 7 Mar 2025 15:15:50 -0300 Message-Id: <20250307181551.19887-8-farosas@suse.de> X-Mailer: git-send-email 2.35.3 In-Reply-To: <20250307181551.19887-1-farosas@suse.de> References: <20250307181551.19887-1-farosas@suse.de> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Spamd-Result: default: False [-2.80 / 50.00]; BAYES_HAM(-3.00)[100.00%]; MID_CONTAINS_FROM(1.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; R_MISSING_CHARSET(0.50)[]; NEURAL_HAM_SHORT(-0.20)[-0.999]; MIME_GOOD(-0.10)[text/plain]; MIME_TRACE(0.00)[0:+]; TO_DN_SOME(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; ARC_NA(0.00)[]; DKIM_SIGNED(0.00)[suse.de:s=susede2_rsa,suse.de:s=susede2_ed25519]; FUZZY_BLOCKED(0.00)[rspamd.com]; FROM_EQ_ENVFROM(0.00)[]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; RCVD_COUNT_TWO(0.00)[2]; TO_MATCH_ENVRCPT_ALL(0.00)[]; DBL_BLOCKED_OPENRESOLVER(0.00)[imap1.dmz-prg2.suse.org:helo,suse.de:email,suse.de:mid]; RCVD_TLS_ALL(0.00)[] X-Spam-Score: -2.80 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=195.135.223.130; envelope-from=farosas@suse.de; helo=smtp-out1.suse.de X-Spam_score_int: -43 X-Spam_score: -4.4 X-Spam_bar: ---- X-Spam_report: (-4.4 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, RCVD_IN_VALIDITY_SAFE_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @suse.de) X-ZM-MESSAGEID: 1741371513721019100 Content-Type: text/plain; charset="utf-8" From: Li Zhijian control_save_page() is for RDMA only, unfold it to make the code more clear. In addition: - Similar to other branches style in ram_save_target_page(), involve RDMA only if the condition 'migrate_rdma()' is true. - Further simplify the code by removing the RAM_SAVE_CONTROL_NOT_SUPP. Reviewed-by: Peter Xu Signed-off-by: Li Zhijian Message-ID: <20250305062825.772629-6-lizhijian@fujitsu.com> Signed-off-by: Fabiano Rosas --- migration/ram.c | 34 +++++++--------------------------- migration/rdma.c | 7 ++----- migration/rdma.h | 3 +-- 3 files changed, 10 insertions(+), 34 deletions(-) diff --git a/migration/ram.c b/migration/ram.c index 424df6d9f1..c363034c88 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -1143,32 +1143,6 @@ static int save_zero_page(RAMState *rs, PageSearchSt= atus *pss, return len; } =20 -/* - * @pages: the number of pages written by the control path, - * < 0 - error - * > 0 - number of pages written - * - * Return true if the pages has been saved, otherwise false is returned. - */ -static bool control_save_page(PageSearchStatus *pss, - ram_addr_t offset, int *pages) -{ - int ret; - - ret =3D rdma_control_save_page(pss->pss_channel, pss->block->offset, o= ffset, - TARGET_PAGE_SIZE); - if (ret =3D=3D RAM_SAVE_CONTROL_NOT_SUPP) { - return false; - } - - if (ret =3D=3D RAM_SAVE_CONTROL_DELAYED) { - *pages =3D 1; - return true; - } - *pages =3D ret; - return true; -} - /* * directly send the page to the stream * @@ -1965,7 +1939,13 @@ static int ram_save_target_page(RAMState *rs, PageSe= archStatus *pss) int res; =20 /* Hand over to RDMA first */ - if (control_save_page(pss, offset, &res)) { + if (migrate_rdma()) { + res =3D rdma_control_save_page(pss->pss_channel, pss->block->offse= t, + offset, TARGET_PAGE_SIZE); + + if (res =3D=3D RAM_SAVE_CONTROL_DELAYED) { + res =3D 1; + } return res; } =20 diff --git a/migration/rdma.c b/migration/rdma.c index e5b4ac599b..08eb924ffa 100644 --- a/migration/rdma.c +++ b/migration/rdma.c @@ -3284,14 +3284,11 @@ err: int rdma_control_save_page(QEMUFile *f, ram_addr_t block_offset, ram_addr_t offset, size_t size) { - if (!migrate_rdma()) { - return RAM_SAVE_CONTROL_NOT_SUPP; - } + assert(migrate_rdma()); =20 int ret =3D qemu_rdma_save_page(f, block_offset, offset, size); =20 - if (ret !=3D RAM_SAVE_CONTROL_DELAYED && - ret !=3D RAM_SAVE_CONTROL_NOT_SUPP) { + if (ret !=3D RAM_SAVE_CONTROL_DELAYED) { if (ret < 0) { qemu_file_set_error(f, ret); } diff --git a/migration/rdma.h b/migration/rdma.h index f55f28bbed..8eeb0117b9 100644 --- a/migration/rdma.h +++ b/migration/rdma.h @@ -33,7 +33,6 @@ void rdma_start_incoming_migration(InetSocketAddress *hos= t_port, Error **errp); #define RAM_CONTROL_ROUND 1 #define RAM_CONTROL_FINISH 3 =20 -#define RAM_SAVE_CONTROL_NOT_SUPP -1000 #define RAM_SAVE_CONTROL_DELAYED -2000 =20 #ifdef CONFIG_RDMA @@ -56,7 +55,7 @@ static inline int rdma_control_save_page(QEMUFile *f, ram_addr_t block_offset, ram_addr_t offset, size_t size) { - return RAM_SAVE_CONTROL_NOT_SUPP; + g_assert_not_reached(); } #endif #endif --=20 2.35.3 From nobody Sun Nov 16 09:22:51 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=suse.de ARC-Seal: i=1; a=rsa-sha256; t=1741371423; cv=none; d=zohomail.com; s=zohoarc; b=lJnxK2OGQdH2735fPRxBTATixblZIVErR7BnDCliOX0VGpc1HVmsyyaDQDbY/oXiWvfyMui4+G+aDZ0WPhJBEJUkqnHH9tkzYBUPCPMacJHdswV2atzYW1wk+rLkiZliz41UFMpFDkSVnz4wqRDL5QZgnSkiyLorG2XzzV9tNl4= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1741371423; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=nitnp15hL2jUHMEDBolqcEtudwr+GxP9TWOY8/DrRj0=; b=O22X9adoTwvJJ7lNuGr1QiWrDQnNcS2z+kicS+0lwEszqoIrV/HOeHS6uVwnwGNhb+vSbDf5vYfwwi2ppSH/faX/2/VrgkK4cEhh7YmZr+CwGkC+bcb6et2HT/mNSlxUdUIZjOU81o0z5yscCqwdwWFVaClsmeCNVmirGvjAGHY= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1741371423444897.95403851081; Fri, 7 Mar 2025 10:17:03 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1tqcFB-0000kj-B2; Fri, 07 Mar 2025 13:16:21 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tqcFA-0000k7-8U for qemu-devel@nongnu.org; Fri, 07 Mar 2025 13:16:20 -0500 Received: from smtp-out2.suse.de ([195.135.223.131]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1tqcF8-0004x9-22 for qemu-devel@nongnu.org; Fri, 07 Mar 2025 13:16:19 -0500 Received: from imap1.dmz-prg2.suse.org (unknown [10.150.64.97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id 015831F395; Fri, 7 Mar 2025 18:16:11 +0000 (UTC) Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id A122513A22; Fri, 7 Mar 2025 18:16:09 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id yJqyF+k3y2dNcgAAD6G6ig (envelope-from ); Fri, 07 Mar 2025 18:16:09 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1741371371; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=nitnp15hL2jUHMEDBolqcEtudwr+GxP9TWOY8/DrRj0=; b=DWOr1IaAaNxiiax4BcvKMMsctEPDz8MTyvgyBxPvkgQxmmaNlQ3WPjj3AocSTpZKErwqHr G7c804hmrF7B6DOG5sJvZ7pa+MBhEzqYVkCMiiPu67B5fvEs0WqIf8IV6PriwfZhWXrQhV dQFJQhC90dno+QV5tAVyHaal6H0x+4g= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1741371371; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=nitnp15hL2jUHMEDBolqcEtudwr+GxP9TWOY8/DrRj0=; b=O7GCwntSicCRuA3fvmD4bY3zxJ5nO08U45G7IFIy+ula/HX/Dko09vaNEsU7w/R5d6oOVT dVi63Guna5hfVsAQ== Authentication-Results: smtp-out2.suse.de; none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1741371371; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=nitnp15hL2jUHMEDBolqcEtudwr+GxP9TWOY8/DrRj0=; b=DWOr1IaAaNxiiax4BcvKMMsctEPDz8MTyvgyBxPvkgQxmmaNlQ3WPjj3AocSTpZKErwqHr G7c804hmrF7B6DOG5sJvZ7pa+MBhEzqYVkCMiiPu67B5fvEs0WqIf8IV6PriwfZhWXrQhV dQFJQhC90dno+QV5tAVyHaal6H0x+4g= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1741371371; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=nitnp15hL2jUHMEDBolqcEtudwr+GxP9TWOY8/DrRj0=; b=O7GCwntSicCRuA3fvmD4bY3zxJ5nO08U45G7IFIy+ula/HX/Dko09vaNEsU7w/R5d6oOVT dVi63Guna5hfVsAQ== From: Fabiano Rosas To: qemu-devel@nongnu.org Cc: Peter Xu , Li Zhijian Subject: [PULL 8/8] migration: Add qtest for migration over RDMA Date: Fri, 7 Mar 2025 15:15:51 -0300 Message-Id: <20250307181551.19887-9-farosas@suse.de> X-Mailer: git-send-email 2.35.3 In-Reply-To: <20250307181551.19887-1-farosas@suse.de> References: <20250307181551.19887-1-farosas@suse.de> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Spamd-Result: default: False [-2.80 / 50.00]; BAYES_HAM(-3.00)[100.00%]; MID_CONTAINS_FROM(1.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; R_MISSING_CHARSET(0.50)[]; NEURAL_HAM_SHORT(-0.20)[-0.999]; MIME_GOOD(-0.10)[text/plain]; MIME_TRACE(0.00)[0:+]; TO_DN_SOME(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; ARC_NA(0.00)[]; DKIM_SIGNED(0.00)[suse.de:s=susede2_rsa,suse.de:s=susede2_ed25519]; FUZZY_BLOCKED(0.00)[rspamd.com]; FROM_EQ_ENVFROM(0.00)[]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; RCVD_COUNT_TWO(0.00)[2]; TO_MATCH_ENVRCPT_ALL(0.00)[]; DBL_BLOCKED_OPENRESOLVER(0.00)[imap1.dmz-prg2.suse.org:helo,suse.de:email,suse.de:mid]; RCVD_TLS_ALL(0.00)[] X-Spam-Score: -2.80 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=195.135.223.131; envelope-from=farosas@suse.de; helo=smtp-out2.suse.de X-Spam_score_int: -43 X-Spam_score: -4.4 X-Spam_bar: ---- X-Spam_report: (-4.4 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, RCVD_IN_VALIDITY_SAFE_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @suse.de) X-ZM-MESSAGEID: 1741371425594019100 Content-Type: text/plain; charset="utf-8" From: Li Zhijian This qtest requires there is a RDMA(RoCE) link in the host. In order to make the test work smoothly, introduce a scripts/rdma-migration-helper.sh to - setup a new Soft-RoCE(aka RXE) if it's root - detect existing RoCE link Test will be skipped if there is no available RoCE link. # Start of rdma tests # Running /x86_64/migration/precopy/rdma/plain Command 'rdma' is not available, please install it first. # To enable the test: # (1) Run 'scripts/rdma-migration-helper.sh setup' with root and rerun the= test # or # (2) Run the test with root privilege # ok 1 /x86_64/migration/precopy/rdma/plain # SKIP No rdma link available # End of rdma tests Note: Remove the newly added RXE link by executing 'modprobe -r rdma_rxe' or by specifying 'clean' within this script. Reviewed-by: Peter Xu Signed-off-by: Li Zhijian Message-ID: <20250305062825.772629-7-lizhijian@fujitsu.com> [reformated the message to be under 90 characters] Signed-off-by: Fabiano Rosas --- MAINTAINERS | 1 + scripts/rdma-migration-helper.sh | 48 +++++++++++++++++++ tests/qtest/migration/precopy-tests.c | 69 +++++++++++++++++++++++++++ 3 files changed, 118 insertions(+) create mode 100755 scripts/rdma-migration-helper.sh diff --git a/MAINTAINERS b/MAINTAINERS index 5df6020ed5..56e85adcfb 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -3517,6 +3517,7 @@ R: Li Zhijian R: Peter Xu S: Odd Fixes F: migration/rdma* +F: scripts/rdma-migration-helper.sh =20 Migration dirty limit and dirty page rate M: Hyman Huang diff --git a/scripts/rdma-migration-helper.sh b/scripts/rdma-migration-help= er.sh new file mode 100755 index 0000000000..08e29a52eb --- /dev/null +++ b/scripts/rdma-migration-helper.sh @@ -0,0 +1,48 @@ +#!/bin/bash + +# Copied from blktests +get_ipv4_addr() +{ + ip -4 -o addr show dev "$1" | + sed -n 's/.*[[:blank:]]inet[[:blank:]]*\([^[:blank:]/]*\).*/\1/p' | + tr -d '\n' +} + +has_soft_rdma() +{ + rdma link | grep -q " netdev $1[[:blank:]]*\$" +} + +rdma_rxe_setup_detect() +{ + ( + cd /sys/class/net && + for i in *; do + [ -e "$i" ] || continue + [ "$i" =3D "lo" ] && continue + [ "$(<"$i/addr_len")" =3D 6 ] || continue + [ "$(<"$i/carrier")" =3D 1 ] || continue + + has_soft_rdma "$i" && break + [ "$operation" =3D "setup" ] && + rdma link add "${i}_rxe" type rxe netdev "$i" && break + done + has_soft_rdma "$i" || return + get_ipv4_addr "$i" + ) +} + +operation=3D${1:-setup} + +command -v rdma >/dev/null || { + echo "Command 'rdma' is not available, please install it first." >&2 + exit 1 +} + +if [ "$operation" =3D=3D "setup" ] || [ "$operation" =3D=3D "detect" ]; th= en + rdma_rxe_setup_detect +elif [ "$operation" =3D=3D "clean" ]; then + modprobe -r rdma_rxe +else + echo "Usage: $0 [setup | detect | clean]" +fi diff --git a/tests/qtest/migration/precopy-tests.c b/tests/qtest/migration/= precopy-tests.c index ba273d10b9..f1fe34020d 100644 --- a/tests/qtest/migration/precopy-tests.c +++ b/tests/qtest/migration/precopy-tests.c @@ -99,6 +99,71 @@ static void test_precopy_unix_dirty_ring(void) test_precopy_common(&args); } =20 +#ifdef CONFIG_RDMA + +#define RDMA_MIGRATION_HELPER "scripts/rdma-migration-helper.sh" +static int new_rdma_link(char *buffer, bool verbose) +{ + const char *argument =3D (geteuid() =3D=3D 0) ? "setup" : "detect"; + char cmd[1024]; + + snprintf(cmd, sizeof(cmd), "%s %s %s", RDMA_MIGRATION_HELPER, argument, + verbose ? "" : "2>/dev/null"); + + FILE *pipe =3D popen(cmd, "r"); + if (pipe =3D=3D NULL) { + perror("Failed to run script"); + return -1; + } + + int idx =3D 0; + while (fgets(buffer + idx, 128 - idx, pipe) !=3D NULL) { + idx +=3D strlen(buffer); + } + + int status =3D pclose(pipe); + if (status =3D=3D -1) { + perror("Error reported by pclose()"); + return -1; + } else if (WIFEXITED(status)) { + return WEXITSTATUS(status); + } + + return -1; +} + +static void test_precopy_rdma_plain(void) +{ + char buffer[128] =3D {}; + bool verbose =3D g_getenv("QTEST_LOG"); + + if (new_rdma_link(buffer, verbose)) { + g_test_skip("No rdma link available"); + if (verbose) { + g_test_message( + "To enable the test:\n" + "(1) Run \'" RDMA_MIGRATION_HELPER + " setup\' with root and rerun the test\n" + "or\n(2) Run the test with root privilege"); + } + return; + } + + /* + * TODO: query a free port instead of hard code. + * 29200=3D('R'+'D'+'M'+'A')*100 + **/ + g_autofree char *uri =3D g_strdup_printf("rdma:%s:29200", buffer); + + MigrateCommon args =3D { + .listen_uri =3D uri, + .connect_uri =3D uri, + }; + + test_precopy_common(&args); +} +#endif + static void test_precopy_tcp_plain(void) { MigrateCommon args =3D { @@ -1124,6 +1189,10 @@ static void migration_test_add_precopy_smoke(Migrati= onTestEnv *env) test_multifd_tcp_uri_none); migration_test_add("/migration/multifd/tcp/plain/cancel", test_multifd_tcp_cancel); +#ifdef CONFIG_RDMA + migration_test_add("/migration/precopy/rdma/plain", + test_precopy_rdma_plain); +#endif } =20 void migration_test_add_precopy(MigrationTestEnv *env) --=20 2.35.3