From nobody Mon Feb 9 23:00:46 2026 Delivered-To: importer@patchew.org Received-SPF: none (zohomail.com: 192.237.175.120 is neither permitted nor denied by domain of lists.xenproject.org) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; dkim=fail; spf=none (zohomail.com: 192.237.175.120 is neither permitted nor denied by domain of lists.xenproject.org) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=fail(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1582303492; cv=none; d=zohomail.com; s=zohoarc; b=HMUogqo0AEqHdECIFcuNinrlYUY3txAATrFsGzWnor9RduRKyX9kErapowDJOUBP6e/YZzE3kFW0e5IINS/stlDakCfNi1/CRK//JoAwBdn69SaHC6vnqDF05BtRPmecwrQj5p6swo1PQsyihkJm67MtbQUQf8xDLVtww/U6kCQ= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1582303492; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=fQjzCqZhARSYHJQZf3KEM3mOuM2+BLabmqtAJozpiYY=; b=NAxQuHRip5RpbLmWOXrFWjFDyUzMzm5xn4Z0xjRg64vOUT8ljCzZsPSdsP57zsQAIBCr9zZ2WH0vqGnkP0bkDonCO7dspdKT4lqEyuPayNI3VXAwfkyzVCJ3DkoFuVqOYUlVY0jgObjoReBIpKqf+R48mO3xH198zaRuuWk50zU= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=fail; spf=none (zohomail.com: 192.237.175.120 is neither permitted nor denied by domain of lists.xenproject.org) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1582303492653366.00937703686657; Fri, 21 Feb 2020 08:44:52 -0800 (PST) Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1j5BPU-0002F1-KD; Fri, 21 Feb 2020 16:44:16 +0000 Received: from all-amaz-eas1.inumbo.com ([34.197.232.57] helo=us1-amaz-eas2.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1j5BPT-0002Ek-6H for xen-devel@lists.xenproject.org; Fri, 21 Feb 2020 16:44:15 +0000 Received: from us-smtp-1.mimecast.com (unknown [205.139.110.61]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTP id 3f9705f7-54c9-11ea-86b2-12813bfff9fa; Fri, 21 Feb 2020 16:43:12 +0000 (UTC) Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-14-MWTpaPyAObe9_oE9y9YcQA-1; Fri, 21 Feb 2020 11:43:10 -0500 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 92965801A0E; Fri, 21 Feb 2020 16:43:08 +0000 (UTC) Received: from t480s.redhat.com (ovpn-117-197.ams2.redhat.com [10.36.117.197]) by smtp.corp.redhat.com (Postfix) with ESMTP id F3FAE5C105; Fri, 21 Feb 2020 16:43:02 +0000 (UTC) X-Inumbo-ID: 3f9705f7-54c9-11ea-86b2-12813bfff9fa DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1582303392; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Gm4E2H+UsvDbJoOz38xrcXlXvsFFgBUsWOd4QRP/Pc8=; b=G3jxDonSVoYb+cZL10n86sMHcIEMnUiOvJlqkOYMqhWVt5gv8iupiJqEpFErxM3QWye8dt jdZJqrTMTjt5VjrCqLO/qaKIZJqjFPbZDrkvj1qcXdPiw/QTdA/7ChLoAEEU0GW/f1U/QY yUYJymKqUOflPuq27wvcU93mpvLQRbw= X-MC-Unique: MWTpaPyAObe9_oE9y9YcQA-1 From: David Hildenbrand To: qemu-devel@nongnu.org Date: Fri, 21 Feb 2020 17:41:54 +0100 Message-Id: <20200221164204.105570-4-david@redhat.com> In-Reply-To: <20200221164204.105570-1-david@redhat.com> References: <20200221164204.105570-1-david@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Subject: [Xen-devel] [PATCH v2 03/13] numa: Teach ram block notifiers about resizeable ram blocks X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Stefano Stabellini , Eduardo Habkost , Juan Quintela , David Hildenbrand , "Dr . David Alan Gilbert" , Peter Xu , Paul Durrant , Igor Mammedov , "Michael S. Tsirkin" , Marcel Apfelbaum , xen-devel@lists.xenproject.org, Anthony Perard , Paolo Bonzini , Richard Henderson Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) Ram block notifiers are currently not aware of resizes. Especially to handle resizes during migration, but also to implement actually resizeable ram blocks (make everything between used_length and max_length inaccessible), we want to teach ram block notifiers about resizeable ram. Introduce the basic infrastructure but keep using max_size in the existing notifiers. Supply the max_size when adding and removing ram blocks. Also, notify on resizes. Acked-by: Paul Durrant Reviewed-by: Peter Xu Cc: Richard Henderson Cc: Paolo Bonzini Cc: "Dr. David Alan Gilbert" Cc: Eduardo Habkost Cc: Marcel Apfelbaum Cc: Stefano Stabellini Cc: Anthony Perard Cc: Paul Durrant Cc: "Michael S. Tsirkin" Cc: xen-devel@lists.xenproject.org Cc: Igor Mammedov Signed-off-by: David Hildenbrand --- exec.c | 13 +++++++++++-- hw/core/numa.c | 22 +++++++++++++++++----- hw/i386/xen/xen-mapcache.c | 7 ++++--- include/exec/ramlist.h | 13 +++++++++---- target/i386/hax-mem.c | 5 +++-- target/i386/sev.c | 18 ++++++++++-------- util/vfio-helpers.c | 16 ++++++++-------- 7 files changed, 62 insertions(+), 32 deletions(-) diff --git a/exec.c b/exec.c index dfd43d27c6..b75250e773 100644 --- a/exec.c +++ b/exec.c @@ -2129,6 +2129,8 @@ static int memory_try_enable_merging(void *addr, size= _t len) */ int qemu_ram_resize(RAMBlock *block, ram_addr_t newsize, Error **errp) { + const ram_addr_t oldsize =3D block->used_length; + assert(block); =20 newsize =3D HOST_PAGE_ALIGN(newsize); @@ -2153,6 +2155,11 @@ int qemu_ram_resize(RAMBlock *block, ram_addr_t news= ize, Error **errp) return -EINVAL; } =20 + /* Notify before modifying the ram block and touching the bitmaps. */ + if (block->host) { + ram_block_notify_resize(block->host, oldsize, newsize); + } + cpu_physical_memory_clear_dirty_range(block->offset, block->used_lengt= h); block->used_length =3D newsize; cpu_physical_memory_set_dirty_range(block->offset, block->used_length, @@ -2312,7 +2319,8 @@ static void ram_block_add(RAMBlock *new_block, Error = **errp, bool shared) qemu_madvise(new_block->host, new_block->max_length, QEMU_MADV_HUG= EPAGE); /* MADV_DONTFORK is also needed by KVM in absence of synchronous M= MU */ qemu_madvise(new_block->host, new_block->max_length, QEMU_MADV_DON= TFORK); - ram_block_notify_add(new_block->host, new_block->max_length); + ram_block_notify_add(new_block->host, new_block->used_length, + new_block->max_length); } } =20 @@ -2492,7 +2500,8 @@ void qemu_ram_free(RAMBlock *block) } =20 if (block->host) { - ram_block_notify_remove(block->host, block->max_length); + ram_block_notify_remove(block->host, block->used_length, + block->max_length); } =20 qemu_mutex_lock_ramlist(); diff --git a/hw/core/numa.c b/hw/core/numa.c index 6599c69e05..e28ad24fcd 100644 --- a/hw/core/numa.c +++ b/hw/core/numa.c @@ -902,11 +902,12 @@ void query_numa_node_mem(NumaNodeMem node_mem[], Mach= ineState *ms) static int ram_block_notify_add_single(RAMBlock *rb, void *opaque) { const ram_addr_t max_size =3D qemu_ram_get_max_length(rb); + const ram_addr_t size =3D qemu_ram_get_used_length(rb); void *host =3D qemu_ram_get_host_addr(rb); RAMBlockNotifier *notifier =3D opaque; =20 if (host) { - notifier->ram_block_added(notifier, host, max_size); + notifier->ram_block_added(notifier, host, size, max_size); } return 0; } @@ -923,20 +924,31 @@ void ram_block_notifier_remove(RAMBlockNotifier *n) QLIST_REMOVE(n, next); } =20 -void ram_block_notify_add(void *host, size_t size) +void ram_block_notify_add(void *host, size_t size, size_t max_size) { RAMBlockNotifier *notifier; =20 QLIST_FOREACH(notifier, &ram_list.ramblock_notifiers, next) { - notifier->ram_block_added(notifier, host, size); + notifier->ram_block_added(notifier, host, size, max_size); } } =20 -void ram_block_notify_remove(void *host, size_t size) +void ram_block_notify_remove(void *host, size_t size, size_t max_size) { RAMBlockNotifier *notifier; =20 QLIST_FOREACH(notifier, &ram_list.ramblock_notifiers, next) { - notifier->ram_block_removed(notifier, host, size); + notifier->ram_block_removed(notifier, host, size, max_size); + } +} + +void ram_block_notify_resize(void *host, size_t old_size, size_t new_size) +{ + RAMBlockNotifier *notifier; + + QLIST_FOREACH(notifier, &ram_list.ramblock_notifiers, next) { + if (notifier->ram_block_resized) { + notifier->ram_block_resized(notifier, host, old_size, new_size= ); + } } } diff --git a/hw/i386/xen/xen-mapcache.c b/hw/i386/xen/xen-mapcache.c index 5b120ed44b..d6dcea65d1 100644 --- a/hw/i386/xen/xen-mapcache.c +++ b/hw/i386/xen/xen-mapcache.c @@ -169,7 +169,8 @@ static void xen_remap_bucket(MapCacheEntry *entry, =20 if (entry->vaddr_base !=3D NULL) { if (!(entry->flags & XEN_MAPCACHE_ENTRY_DUMMY)) { - ram_block_notify_remove(entry->vaddr_base, entry->size); + ram_block_notify_remove(entry->vaddr_base, entry->size, + entry->size); } if (munmap(entry->vaddr_base, entry->size) !=3D 0) { perror("unmap fails"); @@ -211,7 +212,7 @@ static void xen_remap_bucket(MapCacheEntry *entry, } =20 if (!(entry->flags & XEN_MAPCACHE_ENTRY_DUMMY)) { - ram_block_notify_add(vaddr_base, size); + ram_block_notify_add(vaddr_base, size, size); } =20 entry->vaddr_base =3D vaddr_base; @@ -452,7 +453,7 @@ static void xen_invalidate_map_cache_entry_unlocked(uin= t8_t *buffer) } =20 pentry->next =3D entry->next; - ram_block_notify_remove(entry->vaddr_base, entry->size); + ram_block_notify_remove(entry->vaddr_base, entry->size, entry->size); if (munmap(entry->vaddr_base, entry->size) !=3D 0) { perror("unmap fails"); exit(-1); diff --git a/include/exec/ramlist.h b/include/exec/ramlist.h index bc4faa1b00..293c0ddabe 100644 --- a/include/exec/ramlist.h +++ b/include/exec/ramlist.h @@ -65,15 +65,20 @@ void qemu_mutex_lock_ramlist(void); void qemu_mutex_unlock_ramlist(void); =20 struct RAMBlockNotifier { - void (*ram_block_added)(RAMBlockNotifier *n, void *host, size_t size); - void (*ram_block_removed)(RAMBlockNotifier *n, void *host, size_t size= ); + void (*ram_block_added)(RAMBlockNotifier *n, void *host, size_t size, + size_t max_size); + void (*ram_block_removed)(RAMBlockNotifier *n, void *host, size_t size, + size_t max_size); + void (*ram_block_resized)(RAMBlockNotifier *n, void *host, size_t old_= size, + size_t new_size); QLIST_ENTRY(RAMBlockNotifier) next; }; =20 void ram_block_notifier_add(RAMBlockNotifier *n); void ram_block_notifier_remove(RAMBlockNotifier *n); -void ram_block_notify_add(void *host, size_t size); -void ram_block_notify_remove(void *host, size_t size); +void ram_block_notify_add(void *host, size_t size, size_t max_size); +void ram_block_notify_remove(void *host, size_t size, size_t max_size); +void ram_block_notify_resize(void *host, size_t old_size, size_t new_size); =20 void ram_block_dump(Monitor *mon); =20 diff --git a/target/i386/hax-mem.c b/target/i386/hax-mem.c index 6bb5a24917..454d7fb212 100644 --- a/target/i386/hax-mem.c +++ b/target/i386/hax-mem.c @@ -293,7 +293,8 @@ static MemoryListener hax_memory_listener =3D { .priority =3D 10, }; =20 -static void hax_ram_block_added(RAMBlockNotifier *n, void *host, size_t si= ze) +static void hax_ram_block_added(RAMBlockNotifier *n, void *host, size_t si= ze, + size_t max_size) { /* * We must register each RAM block with the HAXM kernel module, or @@ -304,7 +305,7 @@ static void hax_ram_block_added(RAMBlockNotifier *n, vo= id *host, size_t size) * host physical pages for the RAM block as part of this registration * process, hence the name hax_populate_ram(). */ - if (hax_populate_ram((uint64_t)(uintptr_t)host, size) < 0) { + if (hax_populate_ram((uint64_t)(uintptr_t)host, max_size) < 0) { fprintf(stderr, "HAX failed to populate RAM\n"); abort(); } diff --git a/target/i386/sev.c b/target/i386/sev.c index 024bb24e51..6b4cee24a2 100644 --- a/target/i386/sev.c +++ b/target/i386/sev.c @@ -129,7 +129,8 @@ sev_set_guest_state(SevState new_state) } =20 static void -sev_ram_block_added(RAMBlockNotifier *n, void *host, size_t size) +sev_ram_block_added(RAMBlockNotifier *n, void *host, size_t size, + size_t max_size) { int r; struct kvm_enc_region range; @@ -146,19 +147,20 @@ sev_ram_block_added(RAMBlockNotifier *n, void *host, = size_t size) } =20 range.addr =3D (__u64)(unsigned long)host; - range.size =3D size; + range.size =3D max_size; =20 - trace_kvm_memcrypt_register_region(host, size); + trace_kvm_memcrypt_register_region(host, max_size); r =3D kvm_vm_ioctl(kvm_state, KVM_MEMORY_ENCRYPT_REG_REGION, &range); if (r) { error_report("%s: failed to register region (%p+%#zx) error '%s'", - __func__, host, size, strerror(errno)); + __func__, host, max_size, strerror(errno)); exit(1); } } =20 static void -sev_ram_block_removed(RAMBlockNotifier *n, void *host, size_t size) +sev_ram_block_removed(RAMBlockNotifier *n, void *host, size_t size, + size_t max_size) { int r; struct kvm_enc_region range; @@ -175,13 +177,13 @@ sev_ram_block_removed(RAMBlockNotifier *n, void *host= , size_t size) } =20 range.addr =3D (__u64)(unsigned long)host; - range.size =3D size; + range.size =3D max_size; =20 - trace_kvm_memcrypt_unregister_region(host, size); + trace_kvm_memcrypt_unregister_region(host, max_size); r =3D kvm_vm_ioctl(kvm_state, KVM_MEMORY_ENCRYPT_UNREG_REGION, &range); if (r) { error_report("%s: failed to unregister region (%p+%#zx)", - __func__, host, size); + __func__, host, max_size); } } =20 diff --git a/util/vfio-helpers.c b/util/vfio-helpers.c index 260570ae19..9ec01bfe26 100644 --- a/util/vfio-helpers.c +++ b/util/vfio-helpers.c @@ -372,26 +372,26 @@ fail_container: return ret; } =20 -static void qemu_vfio_ram_block_added(RAMBlockNotifier *n, - void *host, size_t size) +static void qemu_vfio_ram_block_added(RAMBlockNotifier *n, void *host, + size_t size, size_t max_size) { QEMUVFIOState *s =3D container_of(n, QEMUVFIOState, ram_notifier); int ret; =20 - trace_qemu_vfio_ram_block_added(s, host, size); - ret =3D qemu_vfio_dma_map(s, host, size, false, NULL); + trace_qemu_vfio_ram_block_added(s, host, max_size); + ret =3D qemu_vfio_dma_map(s, host, max_size, false, NULL); if (ret) { - error_report("qemu_vfio_dma_map(%p, %zu) failed: %s", host, size, + error_report("qemu_vfio_dma_map(%p, %zu) failed: %s", host, max_si= ze, strerror(-ret)); } } =20 -static void qemu_vfio_ram_block_removed(RAMBlockNotifier *n, - void *host, size_t size) +static void qemu_vfio_ram_block_removed(RAMBlockNotifier *n, void *host, + size_t size, size_t max_size) { QEMUVFIOState *s =3D container_of(n, QEMUVFIOState, ram_notifier); if (host) { - trace_qemu_vfio_ram_block_removed(s, host, size); + trace_qemu_vfio_ram_block_removed(s, host, max_size); qemu_vfio_dma_unmap(s, host); } } --=20 2.24.1 _______________________________________________ Xen-devel mailing list Xen-devel@lists.xenproject.org https://lists.xenproject.org/mailman/listinfo/xen-devel