From nobody Tue Dec 16 19:42:57 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=bytedance.com ARC-Seal: i=1; a=rsa-sha256; t=1765826685; cv=none; d=zohomail.com; s=zohoarc; b=nackLd+8Hf5RM1zkc561ArXE0t9agDjg1S5PHhlfkxCMgy6K6TkFYyJbA+V6p+IiehiSnZtN3w5kp9AfPVhG7ecQEZU9xZ5zu3x3aJmQJzxH5CTh2BVKAVspVDjJ9oQ3UnuUcH0iEn4AfDAdcpD71/Aef2dMb2LCebWW9Zjn30Y= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1765826685; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=s0OFiVIW/yPXbHVyol+Lp25JjClNpszFFXFvE6ZWPt8=; b=VL2w9woYqBrHFslBIO3NVoZVxYfrjg4etbb2epHKUgtssFhjnLSOeQ6+SFyxpqj6i/LoAx4f/fr7VjnnaRJ1gLYqRjgXE19wNdLylDuWgPP6owD1gMBnyY/HtSEFFpBEXkQdYaoul8J9gMFJ9PQqc+fX7iKgAt4l0JEcncvMDwE= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1765826685030146.2282314121718; Mon, 15 Dec 2025 11:24:45 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1vV9Ea-0001KN-MQ; Mon, 15 Dec 2025 09:07:32 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1vV9E3-00016k-1e for qemu-devel@nongnu.org; Mon, 15 Dec 2025 09:06:59 -0500 Received: from sg-1-100.ptr.blmpb.com ([118.26.132.100]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1vV9Dt-0000aq-MO for qemu-devel@nongnu.org; Mon, 15 Dec 2025 09:06:55 -0500 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; s=2212171451; d=bytedance.com; t=1765807594; h=from:subject: mime-version:from:date:message-id:subject:to:cc:reply-to:content-type: mime-version:in-reply-to:message-id; bh=s0OFiVIW/yPXbHVyol+Lp25JjClNpszFFXFvE6ZWPt8=; b=kp41W3aH0+pYg8t5vF+jO3lQ23b6vK/NCd+unJsgDT+XhDqIo1LBWpa2gE1OvoiMT/3jE3 SvrptCaDdcSqmchQojyXTAFCvyBMu6iqmUccQnlsaxK+OhgXqGtzWCKEnsua1GbUn3mbfr e5TX9Okqo5vqsQ/1kvQ8tjlVZ4YaHKu56HqKEehJyzOwZUx72RcI5YUmBlJ+IbRa7rZoAR 2eR1MrR+99/C1v8kTxSYxD3R+E81NBMk0VJpG3G0HmAZfomRRl8x3vaaC/hjQMCBng1UO3 r5fNe8C8n1Rk4MomogaMBUwja4RBd4LV1ZvhDojm02VLpaIsvBy2IR5qyggteg== Subject: [PATCH v2 1/1] migration: merge fragmented clear_dirty ioctls References: <20251215140611.16180-1-xuchuangxclwt@bytedance.com> Cc: , , , , , , , , "xuchuangxclwt" X-Mailer: git-send-email 2.39.3 (Apple Git-146) X-Original-From: Chuang Xu From: "Chuang Xu" Message-Id: <20251215140611.16180-2-xuchuangxclwt@bytedance.com> X-Lms-Return-Path: In-Reply-To: <20251215140611.16180-1-xuchuangxclwt@bytedance.com> Content-Transfer-Encoding: quoted-printable To: Mime-Version: 1.0 Date: Mon, 15 Dec 2025 22:06:11 +0800 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=118.26.132.100; envelope-from=xuchuangxclwt@bytedance.com; helo=sg-1-100.ptr.blmpb.com X-Spam_score_int: -15 X-Spam_score: -1.6 X-Spam_bar: - X-Spam_report: (-1.6 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FROM_LOCAL_NOVOWEL=0.5, HK_RANDOM_ENVFROM=0.001, HK_RANDOM_FROM=0.001, RCVD_IN_VALIDITY_CERTIFIED_BLOCKED=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @bytedance.com) X-ZM-MESSAGEID: 1765826688561158500 Content-Type: text/plain; charset="utf-8" From: xuchuangxclwt When the addresses processed are not aligned, a large number of clear_dirty ioctl occur (e.g. a 4MB misaligned memory can generate 2048 clear_dirty ioctls from two different memory_listener), which increases the time required for bitmap_sync and makes it more difficult for dirty pages to converge. Attempt to merge those fragmented clear_dirty ioctls. Signed-off-by: Chuang Xu --- accel/tcg/cputlb.c | 5 +++-- include/system/physmem.h | 7 ++++--- migration/ram.c | 26 ++++++++++++------------ system/memory.c | 2 +- system/physmem.c | 44 ++++++++++++++++++++++++---------------- 5 files changed, 48 insertions(+), 36 deletions(-) diff --git a/accel/tcg/cputlb.c b/accel/tcg/cputlb.c index fd1606c856..c8827c8b0d 100644 --- a/accel/tcg/cputlb.c +++ b/accel/tcg/cputlb.c @@ -857,8 +857,9 @@ void tlb_flush_page_bits_by_mmuidx_all_cpus_synced(CPUS= tate *src_cpu, void tlb_protect_code(ram_addr_t ram_addr) { physical_memory_test_and_clear_dirty(ram_addr & TARGET_PAGE_MASK, - TARGET_PAGE_SIZE, - DIRTY_MEMORY_CODE); + TARGET_PAGE_SIZE, + DIRTY_MEMORY_CODE, + NULL); } =20 /* update the TLB so that writes in physical page 'phys_addr' are no longer diff --git a/include/system/physmem.h b/include/system/physmem.h index 879f6eae38..8eeace9d1f 100644 --- a/include/system/physmem.h +++ b/include/system/physmem.h @@ -39,9 +39,10 @@ uint64_t physical_memory_set_dirty_lebitmap(unsigned lon= g *bitmap, =20 void physical_memory_dirty_bits_cleared(ram_addr_t start, ram_addr_t lengt= h); =20 -bool physical_memory_test_and_clear_dirty(ram_addr_t start, - ram_addr_t length, - unsigned client); +uint64_t physical_memory_test_and_clear_dirty(ram_addr_t start, + ram_addr_t length, + unsigned client, + unsigned long *dest); =20 DirtyBitmapSnapshot * physical_memory_snapshot_and_clear_dirty(MemoryRegion *mr, hwaddr offset, diff --git a/migration/ram.c b/migration/ram.c index 29f016cb25..2d5e979211 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -942,7 +942,6 @@ static uint64_t physical_memory_sync_dirty_bitmap(RAMBl= ock *rb, ram_addr_t start, ram_addr_t length) { - ram_addr_t addr; unsigned long word =3D BIT_WORD((start + rb->offset) >> TARGET_PAGE_BI= TS); uint64_t num_dirty =3D 0; unsigned long *dest =3D rb->bmap; @@ -995,18 +994,19 @@ static uint64_t physical_memory_sync_dirty_bitmap(RAM= Block *rb, } } else { ram_addr_t offset =3D rb->offset; - - for (addr =3D 0; addr < length; addr +=3D TARGET_PAGE_SIZE) { - if (physical_memory_test_and_clear_dirty( - start + addr + offset, - TARGET_PAGE_SIZE, - DIRTY_MEMORY_MIGRATION)) { - long k =3D (start + addr) >> TARGET_PAGE_BITS; - if (!test_and_set_bit(k, dest)) { - num_dirty++; - } - } - } + unsigned long end, start_page; + uint64_t mr_offset, mr_size; + + num_dirty =3D physical_memory_test_and_clear_dirty( + start + offset, + length, + DIRTY_MEMORY_MIGRATION, + dest); + end =3D TARGET_PAGE_ALIGN(start + offset + length) >> TARGET_PAGE_= BITS; + start_page =3D (start + offset) >> TARGET_PAGE_BITS; + mr_offset =3D (ram_addr_t)(start_page << TARGET_PAGE_BITS) - offse= t; + mr_size =3D (end - start_page) << TARGET_PAGE_BITS; + memory_region_clear_dirty_bitmap(rb->mr, mr_offset, mr_size); } =20 return num_dirty; diff --git a/system/memory.c b/system/memory.c index 8b84661ae3..666364392d 100644 --- a/system/memory.c +++ b/system/memory.c @@ -2424,7 +2424,7 @@ void memory_region_reset_dirty(MemoryRegion *mr, hwad= dr addr, { assert(mr->ram_block); physical_memory_test_and_clear_dirty( - memory_region_get_ram_addr(mr) + addr, size, client); + memory_region_get_ram_addr(mr) + addr, size, client, NULL); } =20 int memory_region_get_fd(MemoryRegion *mr) diff --git a/system/physmem.c b/system/physmem.c index c9869e4049..d015eb2133 100644 --- a/system/physmem.c +++ b/system/physmem.c @@ -1090,18 +1090,19 @@ void physical_memory_set_dirty_range(ram_addr_t sta= rt, ram_addr_t length, } =20 /* Note: start and end must be within the same ram block. */ -bool physical_memory_test_and_clear_dirty(ram_addr_t start, +uint64_t physical_memory_test_and_clear_dirty(ram_addr_t start, ram_addr_t length, - unsigned client) + unsigned client, + unsigned long *dest) { DirtyMemoryBlocks *blocks; unsigned long end, page, start_page; - bool dirty =3D false; + uint64_t num_dirty =3D 0; RAMBlock *ramblock; uint64_t mr_offset, mr_size; =20 if (length =3D=3D 0) { - return false; + return 0; } =20 end =3D TARGET_PAGE_ALIGN(start + length) >> TARGET_PAGE_BITS; @@ -1118,31 +1119,40 @@ bool physical_memory_test_and_clear_dirty(ram_addr_= t start, while (page < end) { unsigned long idx =3D page / DIRTY_MEMORY_BLOCK_SIZE; unsigned long offset =3D page % DIRTY_MEMORY_BLOCK_SIZE; - unsigned long num =3D MIN(end - page, - DIRTY_MEMORY_BLOCK_SIZE - offset); =20 - dirty |=3D bitmap_test_and_clear_atomic(blocks->blocks[idx], - offset, num); - page +=3D num; + if (bitmap_test_and_clear_atomic(blocks->blocks[idx], offset, = 1)) { + if (dest) { + unsigned long k =3D page - (ramblock->offset >> TARGET= _PAGE_BITS); + if (!test_and_set_bit(k, dest)) { + num_dirty++; + } + } else { + num_dirty++; + } + } + + page++; } =20 - mr_offset =3D (ram_addr_t)(start_page << TARGET_PAGE_BITS) - rambl= ock->offset; - mr_size =3D (end - start_page) << TARGET_PAGE_BITS; - memory_region_clear_dirty_bitmap(ramblock->mr, mr_offset, mr_size); + if (!dest && num_dirty) { + mr_offset =3D (ram_addr_t)(start_page << TARGET_PAGE_BITS) - r= amblock->offset; + mr_size =3D (end - start_page) << TARGET_PAGE_BITS; + memory_region_clear_dirty_bitmap(ramblock->mr, mr_offset, mr_s= ize); + } } =20 - if (dirty) { + if (num_dirty) { physical_memory_dirty_bits_cleared(start, length); } =20 - return dirty; + return num_dirty; } =20 static void physical_memory_clear_dirty_range(ram_addr_t addr, ram_addr_t = length) { - physical_memory_test_and_clear_dirty(addr, length, DIRTY_MEMORY_MIGRAT= ION); - physical_memory_test_and_clear_dirty(addr, length, DIRTY_MEMORY_VGA); - physical_memory_test_and_clear_dirty(addr, length, DIRTY_MEMORY_CODE); + physical_memory_test_and_clear_dirty(addr, length, DIRTY_MEMORY_MIGRAT= ION, NULL); + physical_memory_test_and_clear_dirty(addr, length, DIRTY_MEMORY_VGA, N= ULL); + physical_memory_test_and_clear_dirty(addr, length, DIRTY_MEMORY_CODE, = NULL); } =20 DirtyBitmapSnapshot *physical_memory_snapshot_and_clear_dirty --=20 2.39.3 (Apple Git-146)