From nobody Sat Feb 7 10:08:07 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 50071C001B0 for ; Thu, 13 Jul 2023 14:56:57 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229992AbjGMO44 (ORCPT ); Thu, 13 Jul 2023 10:56:56 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:60974 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231169AbjGMO4w (ORCPT ); Thu, 13 Jul 2023 10:56:52 -0400 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 185082713 for ; Thu, 13 Jul 2023 07:55:59 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1689260159; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=rOmThcBQ38XDQnqrguekPpebm9LB32j86WbUbdRAymk=; b=Dq9u35H0Y80xdKoMwM5mEbpOy45hGuldQ51rqEuKOBgxyAQw61udwklvkdvcrHmjYNnEnQ NWl6EbFJg8EwTU+cF0hLqnobwdQ00widS4dafjim7941kdCHeNBtcSMszG5DNt9b6RDtHT 6/lyNO+9dGlQ/emsLgHqnhxToJjIh9Q= Received: from mimecast-mx02.redhat.com (66.187.233.73 [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-651-j4H8HIhJMpCZyM7rsWcs1g-1; Thu, 13 Jul 2023 10:55:57 -0400 X-MC-Unique: j4H8HIhJMpCZyM7rsWcs1g-1 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.rdu2.redhat.com [10.11.54.5]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 4FD7A29A9D36; Thu, 13 Jul 2023 14:55:57 +0000 (UTC) Received: from t14s.redhat.com (unknown [10.39.192.245]) by smtp.corp.redhat.com (Postfix) with ESMTP id A4780F66D1; Thu, 13 Jul 2023 14:55:55 +0000 (UTC) From: David Hildenbrand To: linux-kernel@vger.kernel.org Cc: linux-mm@kvack.org, virtualization@lists.linux-foundation.org, David Hildenbrand , "Michael S. Tsirkin" , Jason Wang , Xuan Zhuo Subject: [PATCH v1 1/4] virtio-mem: remove unsafe unplug in Big Block Mode (BBM) Date: Thu, 13 Jul 2023 16:55:48 +0200 Message-ID: <20230713145551.2824980-2-david@redhat.com> In-Reply-To: <20230713145551.2824980-1-david@redhat.com> References: <20230713145551.2824980-1-david@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.1 on 10.11.54.5 Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" When "unsafe unplug" is enabled, we don't fake-offline all memory ahead of actual memory offlining using alloc_contig_range(). Instead, we rely on offline_pages() to also perform actual page migration, which might fail or take a very long time. In that case, it's possible to easily run into endless loops that cannot be aborted anymore (as offlining is triggered by a workqueue then): For example, a single (accidentally) permanently unmovable page in ZONE_MOVABLE results in an endless loop. For ZONE_NORMAL, races between isolating the pageblock (and checking for unmovable pages) and concurrent page allocation are possible and similarly result in endless loops. The idea of the unsafe unplug mode was to make it possible to more reliably unplug large memory blocks. However, (a) we really should be tackling that differently, by extending the alloc_contig_range()-based mechanism; and (b) this mode is not the default and as far as I know, it's unused either way. So let's simply get rid of it. Signed-off-by: David Hildenbrand --- drivers/virtio/virtio_mem.c | 51 +++++++++++++++---------------------- 1 file changed, 20 insertions(+), 31 deletions(-) diff --git a/drivers/virtio/virtio_mem.c b/drivers/virtio/virtio_mem.c index 835f6cc2fb66..ed15d2a4bd96 100644 --- a/drivers/virtio/virtio_mem.c +++ b/drivers/virtio/virtio_mem.c @@ -38,11 +38,6 @@ module_param(bbm_block_size, ulong, 0444); MODULE_PARM_DESC(bbm_block_size, "Big Block size in bytes. Default is 0 (auto-detection)."); =20 -static bool bbm_safe_unplug =3D true; -module_param(bbm_safe_unplug, bool, 0444); -MODULE_PARM_DESC(bbm_safe_unplug, - "Use a safe unplug mechanism in BBM, avoiding long/endless loops"); - /* * virtio-mem currently supports the following modes of operation: * @@ -2111,38 +2106,32 @@ static int virtio_mem_bbm_offline_remove_and_unplug= _bb(struct virtio_mem *vm, VIRTIO_MEM_BBM_BB_ADDED)) return -EINVAL; =20 - if (bbm_safe_unplug) { - /* - * Start by fake-offlining all memory. Once we marked the device - * block as fake-offline, all newly onlined memory will - * automatically be kept fake-offline. Protect from concurrent - * onlining/offlining until we have a consistent state. - */ - mutex_lock(&vm->hotplug_mutex); - virtio_mem_bbm_set_bb_state(vm, bb_id, - VIRTIO_MEM_BBM_BB_FAKE_OFFLINE); + /* + * Start by fake-offlining all memory. Once we marked the device + * block as fake-offline, all newly onlined memory will + * automatically be kept fake-offline. Protect from concurrent + * onlining/offlining until we have a consistent state. + */ + mutex_lock(&vm->hotplug_mutex); + virtio_mem_bbm_set_bb_state(vm, bb_id, VIRTIO_MEM_BBM_BB_FAKE_OFFLINE); =20 - for (pfn =3D start_pfn; pfn < end_pfn; pfn +=3D PAGES_PER_SECTION) { - page =3D pfn_to_online_page(pfn); - if (!page) - continue; + for (pfn =3D start_pfn; pfn < end_pfn; pfn +=3D PAGES_PER_SECTION) { + page =3D pfn_to_online_page(pfn); + if (!page) + continue; =20 - rc =3D virtio_mem_fake_offline(pfn, PAGES_PER_SECTION); - if (rc) { - end_pfn =3D pfn; - goto rollback_safe_unplug; - } + rc =3D virtio_mem_fake_offline(pfn, PAGES_PER_SECTION); + if (rc) { + end_pfn =3D pfn; + goto rollback; } - mutex_unlock(&vm->hotplug_mutex); } + mutex_unlock(&vm->hotplug_mutex); =20 rc =3D virtio_mem_bbm_offline_and_remove_bb(vm, bb_id); if (rc) { - if (bbm_safe_unplug) { - mutex_lock(&vm->hotplug_mutex); - goto rollback_safe_unplug; - } - return rc; + mutex_lock(&vm->hotplug_mutex); + goto rollback; } =20 rc =3D virtio_mem_bbm_unplug_bb(vm, bb_id); @@ -2154,7 +2143,7 @@ static int virtio_mem_bbm_offline_remove_and_unplug_b= b(struct virtio_mem *vm, VIRTIO_MEM_BBM_BB_UNUSED); return rc; =20 -rollback_safe_unplug: +rollback: for (pfn =3D start_pfn; pfn < end_pfn; pfn +=3D PAGES_PER_SECTION) { page =3D pfn_to_online_page(pfn); if (!page) --=20 2.41.0 From nobody Sat Feb 7 10:08:07 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id C39BEC0015E for ; Thu, 13 Jul 2023 14:57:15 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232094AbjGMO5O (ORCPT ); Thu, 13 Jul 2023 10:57:14 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:32812 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232185AbjGMO5I (ORCPT ); Thu, 13 Jul 2023 10:57:08 -0400 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 2A7EF2736 for ; Thu, 13 Jul 2023 07:56:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1689260163; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=tvagZqP7r7JhOH2HSmKVW5dMeNgJcBr/tywkzsCuQEE=; b=A7w+xqn0O/UJ5p9zcT/RyhNdgkV2iHLGUc94tQsJvw+HLWijwypNicycYpfr1+0r9Icsgr HbpC11K+zoPidJw4RCSlYBKJyCEW6gizaTXtKZJER+tUISXa/uz1DTXAHbNUGDHwX2T11/ HSgKRIi9Lr0s2UWkVcBZsYrOaqqz/8Q= Received: from mimecast-mx02.redhat.com (66.187.233.73 [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-13-nm4gKlnDNE-L9galCwEr3g-1; Thu, 13 Jul 2023 10:55:59 -0400 X-MC-Unique: nm4gKlnDNE-L9galCwEr3g-1 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.rdu2.redhat.com [10.11.54.5]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 640A63815F7E; Thu, 13 Jul 2023 14:55:59 +0000 (UTC) Received: from t14s.redhat.com (unknown [10.39.192.245]) by smtp.corp.redhat.com (Postfix) with ESMTP id B0225F66D7; Thu, 13 Jul 2023 14:55:57 +0000 (UTC) From: David Hildenbrand To: linux-kernel@vger.kernel.org Cc: linux-mm@kvack.org, virtualization@lists.linux-foundation.org, David Hildenbrand , "Michael S. Tsirkin" , Jason Wang , Xuan Zhuo Subject: [PATCH v1 2/4] virtio-mem: convert most offline_and_remove_memory() errors to -EBUSY Date: Thu, 13 Jul 2023 16:55:49 +0200 Message-ID: <20230713145551.2824980-3-david@redhat.com> In-Reply-To: <20230713145551.2824980-1-david@redhat.com> References: <20230713145551.2824980-1-david@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.1 on 10.11.54.5 Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Just like we do with alloc_contig_range(), let's convert all unknown errors to -EBUSY, but WARN so we can look into the issue. For example, offline_pages() could fail with -EINTR, which would be unexpected in our case. Signed-off-by: David Hildenbrand --- drivers/virtio/virtio_mem.c | 12 ++++++++---- 1 file changed, 8 insertions(+), 4 deletions(-) diff --git a/drivers/virtio/virtio_mem.c b/drivers/virtio/virtio_mem.c index ed15d2a4bd96..1a76ba2bc118 100644 --- a/drivers/virtio/virtio_mem.c +++ b/drivers/virtio/virtio_mem.c @@ -741,11 +741,15 @@ static int virtio_mem_offline_and_remove_memory(struc= t virtio_mem *vm, * immediately instead of waiting. */ virtio_mem_retry(vm); - } else { - dev_dbg(&vm->vdev->dev, - "offlining and removing memory failed: %d\n", rc); + return 0; } - return rc; + dev_dbg(&vm->vdev->dev, "offlining and removing memory failed: %d\n", rc); + /* + * We don't really expect this to fail, because we fake-offlined all + * memory already. But it could fail in corner cases. + */ + WARN_ON_ONCE(rc !=3D -ENOMEM && rc !=3D -EBUSY); + return rc =3D=3D -ENOMEM ? -ENOMEM : -EBUSY; } =20 /* --=20 2.41.0 From nobody Sat Feb 7 10:08:07 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 48D3CC001B0 for ; Thu, 13 Jul 2023 14:57:07 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232164AbjGMO5G (ORCPT ); Thu, 13 Jul 2023 10:57:06 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:32786 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231671AbjGMO5E (ORCPT ); Thu, 13 Jul 2023 10:57:04 -0400 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 2A6262735 for ; Thu, 13 Jul 2023 07:56:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1689260163; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=/51NS5IdbGhAF1WTe4etoP/bWkQWV6fSApbrFXQZ0Fk=; b=MWobhaLhcyFJUy2mHqAAjzdH6YBZ3caMU9yGIKHnLWufmHz61b7aW9klbl21GeDQ7qBGQQ i9ZNdDtXR2M6IOLbT3AcmJ3It6oRmFG6KQ0kAkS9/0WRbvVxq0XDV80Euflh42uUuYHfrF TdCI9JhVQQPYLmoGv9jNfPQQHJsDgU4= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-669-Cnd2LY13MK-Zy4GC5ZFc9w-1; Thu, 13 Jul 2023 10:56:01 -0400 X-MC-Unique: Cnd2LY13MK-Zy4GC5ZFc9w-1 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.rdu2.redhat.com [10.11.54.5]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id C7FA283FC20; Thu, 13 Jul 2023 14:56:00 +0000 (UTC) Received: from t14s.redhat.com (unknown [10.39.192.245]) by smtp.corp.redhat.com (Postfix) with ESMTP id 9F036F66D1; Thu, 13 Jul 2023 14:55:59 +0000 (UTC) From: David Hildenbrand To: linux-kernel@vger.kernel.org Cc: linux-mm@kvack.org, virtualization@lists.linux-foundation.org, David Hildenbrand , "Michael S. Tsirkin" , Jason Wang , Xuan Zhuo Subject: [PATCH v1 3/4] virtio-mem: keep retrying on offline_and_remove_memory() errors in Sub Block Mode (SBM) Date: Thu, 13 Jul 2023 16:55:50 +0200 Message-ID: <20230713145551.2824980-4-david@redhat.com> In-Reply-To: <20230713145551.2824980-1-david@redhat.com> References: <20230713145551.2824980-1-david@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.1 on 10.11.54.5 Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" In case offline_and_remove_memory() fails in SBM, we leave a completely unplugged Linux memory block stick around until we try plugging memory again. We won't try removing that memory block again. offline_and_remove_memory() may, for example, fail if we're racing with another alloc_contig_range() user, if allocating temporary memory fails, or if some memory notifier rejected the offlining request. Let's handle that case better, by simple retrying to offline and remove such memory. Tested using CONFIG_MEMORY_NOTIFIER_ERROR_INJECT. Signed-off-by: David Hildenbrand --- drivers/virtio/virtio_mem.c | 92 +++++++++++++++++++++++++++++-------- 1 file changed, 73 insertions(+), 19 deletions(-) diff --git a/drivers/virtio/virtio_mem.c b/drivers/virtio/virtio_mem.c index 1a76ba2bc118..a5cf92e3e5af 100644 --- a/drivers/virtio/virtio_mem.c +++ b/drivers/virtio/virtio_mem.c @@ -168,6 +168,13 @@ struct virtio_mem { /* The number of subblocks per Linux memory block. */ uint32_t sbs_per_mb; =20 + /* + * Some of the Linux memory blocks tracked as "partially + * plugged" are completely unplugged and can be offlined + * and removed -- which previously failed. + */ + bool have_unplugged_mb; + /* Summary of all memory block states. */ unsigned long mb_count[VIRTIO_MEM_SBM_MB_COUNT]; =20 @@ -765,6 +772,34 @@ static int virtio_mem_sbm_offline_and_remove_mb(struct= virtio_mem *vm, return virtio_mem_offline_and_remove_memory(vm, addr, size); } =20 +/* + * Try (offlining and) removing memory from Linux in case all subblocks are + * unplugged. Can be called on online and offline memory blocks. + * + * May modify the state of memory blocks in virtio-mem. + */ +static int virtio_mem_sbm_try_remove_unplugged_mb(struct virtio_mem *vm, + unsigned long mb_id) +{ + int rc; + + /* + * Once all subblocks of a memory block were unplugged, offline and + * remove it. + */ + if (!virtio_mem_sbm_test_sb_unplugged(vm, mb_id, 0, vm->sbm.sbs_per_mb)) + return 0; + + /* offline_and_remove_memory() works for online and offline memory. */ + mutex_unlock(&vm->hotplug_mutex); + rc =3D virtio_mem_sbm_offline_and_remove_mb(vm, mb_id); + mutex_lock(&vm->hotplug_mutex); + if (!rc) + virtio_mem_sbm_set_mb_state(vm, mb_id, + VIRTIO_MEM_SBM_MB_UNUSED); + return rc; +} + /* * See virtio_mem_offline_and_remove_memory(): Try to offline and remove a * all Linux memory blocks covered by the big block. @@ -1988,20 +2023,10 @@ static int virtio_mem_sbm_unplug_any_sb_online(stru= ct virtio_mem *vm, } =20 unplugged: - /* - * Once all subblocks of a memory block were unplugged, offline and - * remove it. This will usually not fail, as no memory is in use - * anymore - however some other notifiers might NACK the request. - */ - if (virtio_mem_sbm_test_sb_unplugged(vm, mb_id, 0, vm->sbm.sbs_per_mb)) { - mutex_unlock(&vm->hotplug_mutex); - rc =3D virtio_mem_sbm_offline_and_remove_mb(vm, mb_id); - mutex_lock(&vm->hotplug_mutex); - if (!rc) - virtio_mem_sbm_set_mb_state(vm, mb_id, - VIRTIO_MEM_SBM_MB_UNUSED); - } - + rc =3D virtio_mem_sbm_try_remove_unplugged_mb(vm, mb_id); + if (rc) + vm->sbm.have_unplugged_mb =3D 1; + /* Ignore errors, this is not critical. We'll retry later. */ return 0; } =20 @@ -2253,12 +2278,13 @@ static int virtio_mem_unplug_request(struct virtio_= mem *vm, uint64_t diff) =20 /* * Try to unplug all blocks that couldn't be unplugged before, for example, - * because the hypervisor was busy. + * because the hypervisor was busy. Further, offline and remove any memory + * blocks where we previously failed. */ -static int virtio_mem_unplug_pending_mb(struct virtio_mem *vm) +static int virtio_mem_cleanup_pending_mb(struct virtio_mem *vm) { unsigned long id; - int rc; + int rc =3D 0; =20 if (!vm->in_sbm) { virtio_mem_bbm_for_each_bb(vm, id, @@ -2280,6 +2306,27 @@ static int virtio_mem_unplug_pending_mb(struct virti= o_mem *vm) VIRTIO_MEM_SBM_MB_UNUSED); } =20 + if (!vm->sbm.have_unplugged_mb) + return 0; + + /* + * Let's retry (offlining and) removing completely unplugged Linux + * memory blocks. + */ + vm->sbm.have_unplugged_mb =3D false; + + mutex_lock(&vm->hotplug_mutex); + virtio_mem_sbm_for_each_mb(vm, id, VIRTIO_MEM_SBM_MB_MOVABLE_PARTIAL) + rc |=3D virtio_mem_sbm_try_remove_unplugged_mb(vm, id); + virtio_mem_sbm_for_each_mb(vm, id, VIRTIO_MEM_SBM_MB_KERNEL_PARTIAL) + rc |=3D virtio_mem_sbm_try_remove_unplugged_mb(vm, id); + virtio_mem_sbm_for_each_mb(vm, id, VIRTIO_MEM_SBM_MB_OFFLINE_PARTIAL) + rc |=3D virtio_mem_sbm_try_remove_unplugged_mb(vm, id); + mutex_unlock(&vm->hotplug_mutex); + + if (rc) + vm->sbm.have_unplugged_mb =3D true; + /* Ignore errors, this is not critical. We'll retry later. */ return 0; } =20 @@ -2361,9 +2408,9 @@ static void virtio_mem_run_wq(struct work_struct *wor= k) virtio_mem_refresh_config(vm); } =20 - /* Unplug any leftovers from previous runs */ + /* Cleanup any leftovers from previous runs */ if (!rc) - rc =3D virtio_mem_unplug_pending_mb(vm); + rc =3D virtio_mem_cleanup_pending_mb(vm); =20 if (!rc && vm->requested_size !=3D vm->plugged_size) { if (vm->requested_size > vm->plugged_size) { @@ -2375,6 +2422,13 @@ static void virtio_mem_run_wq(struct work_struct *wo= rk) } } =20 + /* + * Keep retrying to offline and remove completely unplugged Linux + * memory blocks. + */ + if (!rc && vm->in_sbm && vm->sbm.have_unplugged_mb) + rc =3D -EBUSY; + switch (rc) { case 0: vm->retry_timer_ms =3D VIRTIO_MEM_RETRY_TIMER_MIN_MS; --=20 2.41.0 From nobody Sat Feb 7 10:08:07 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3E420C001B0 for ; Thu, 13 Jul 2023 14:57:14 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232256AbjGMO5N (ORCPT ); Thu, 13 Jul 2023 10:57:13 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:60978 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231671AbjGMO5H (ORCPT ); Thu, 13 Jul 2023 10:57:07 -0400 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0A5AA2D45 for ; Thu, 13 Jul 2023 07:56:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1689260166; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Bk+toP/Ed4qPpAgMLWeptgvuADn7go/7Jo6jlttF96s=; b=gNSwROucsZMlcoBwn+p2eERWuaXeVuw+ri+Gq2TpGXSI3q7kzYg33kF9tQXZx3K5T6J9dH NC/cKna0b6CNW7GLFa17Rt/WRrMgnBOuXgOfSbZTPPNWhXCER+4RI5KaUX8ddsaXfqsO0i IBJ94gFqHvglentirGpwYhLealUd3l4= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-351-pEnhlwaAPsitRYud8qL9ig-1; Thu, 13 Jul 2023 10:56:02 -0400 X-MC-Unique: pEnhlwaAPsitRYud8qL9ig-1 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.rdu2.redhat.com [10.11.54.5]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 58A8086F122; Thu, 13 Jul 2023 14:56:02 +0000 (UTC) Received: from t14s.redhat.com (unknown [10.39.192.245]) by smtp.corp.redhat.com (Postfix) with ESMTP id 197A0F66CB; Thu, 13 Jul 2023 14:56:00 +0000 (UTC) From: David Hildenbrand To: linux-kernel@vger.kernel.org Cc: linux-mm@kvack.org, virtualization@lists.linux-foundation.org, David Hildenbrand , "Michael S. Tsirkin" , Jason Wang , Xuan Zhuo Subject: [PATCH v1 4/4] virtio-mem: check if the config changed before fake offlining memory Date: Thu, 13 Jul 2023 16:55:51 +0200 Message-ID: <20230713145551.2824980-5-david@redhat.com> In-Reply-To: <20230713145551.2824980-1-david@redhat.com> References: <20230713145551.2824980-1-david@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.1 on 10.11.54.5 Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" If we repeatedly fail to fake offline memory to unplug it, we won't be sending any unplug requests to the device. However, we only check if the config changed when sending such (un)plug requests. We could end up trying for a long time to unplug memory, even though the config changed already and we're not supposed to unplug memory anymore. For example, the hypervisor might detect a low-memory situation while unplugging memory and decide to replug some memory. Continuing trying to unplug memory in that case can be problematic. So let's check on a more regular basis. Signed-off-by: David Hildenbrand --- drivers/virtio/virtio_mem.c | 15 ++++++++++++--- 1 file changed, 12 insertions(+), 3 deletions(-) diff --git a/drivers/virtio/virtio_mem.c b/drivers/virtio/virtio_mem.c index a5cf92e3e5af..fa5226c198cc 100644 --- a/drivers/virtio/virtio_mem.c +++ b/drivers/virtio/virtio_mem.c @@ -1189,7 +1189,8 @@ static void virtio_mem_fake_online(unsigned long pfn,= unsigned long nr_pages) * Try to allocate a range, marking pages fake-offline, effectively * fake-offlining them. */ -static int virtio_mem_fake_offline(unsigned long pfn, unsigned long nr_pag= es) +static int virtio_mem_fake_offline(struct virtio_mem *vm, unsigned long pf= n, + unsigned long nr_pages) { const bool is_movable =3D is_zone_movable_page(pfn_to_page(pfn)); int rc, retry_count; @@ -1202,6 +1203,14 @@ static int virtio_mem_fake_offline(unsigned long pfn= , unsigned long nr_pages) * some guarantees. */ for (retry_count =3D 0; retry_count < 5; retry_count++) { + /* + * If the config changed, stop immediately and go back to the + * main loop: avoid trying to keep unplugging if the device + * might have decided to not remove any more memory. + */ + if (atomic_read(&vm->config_changed)) + return -EAGAIN; + rc =3D alloc_contig_range(pfn, pfn + nr_pages, MIGRATE_MOVABLE, GFP_KERNEL); if (rc =3D=3D -ENOMEM) @@ -1951,7 +1960,7 @@ static int virtio_mem_sbm_unplug_sb_online(struct vir= tio_mem *vm, start_pfn =3D PFN_DOWN(virtio_mem_mb_id_to_phys(mb_id) + sb_id * vm->sbm.sb_size); =20 - rc =3D virtio_mem_fake_offline(start_pfn, nr_pages); + rc =3D virtio_mem_fake_offline(vm, start_pfn, nr_pages); if (rc) return rc; =20 @@ -2149,7 +2158,7 @@ static int virtio_mem_bbm_offline_remove_and_unplug_b= b(struct virtio_mem *vm, if (!page) continue; =20 - rc =3D virtio_mem_fake_offline(pfn, PAGES_PER_SECTION); + rc =3D virtio_mem_fake_offline(vm, pfn, PAGES_PER_SECTION); if (rc) { end_pfn =3D pfn; goto rollback; --=20 2.41.0