From nobody Sun Apr 28 23:35:52 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1569340082; cv=none; d=zoho.com; s=zohoarc; b=kRgxNGvA1mrHh1VWNiHK7ignfPjZpEDOEvdiBg7r0co3qy3ctZbIh5DIsgl3OK5q5SUehQ0OiQDsctJtOSqo6UL5GSVQYIkQaZWDvHRRIJbyRXi80GBTu65z39802hSHefFMhO5KLii0/94eEzIcdxAnBXJqKaToevGj1WWHyA8= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1569340082; h=Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:Message-ID:References:Sender:Subject:To:ARC-Authentication-Results; bh=GgGw6HTDfTEtryWdX6dcR1UoLci65sBH1p9yKZOB7+E=; b=WVetIvgbS7///svVQRZXhKGI0XF1TMSoJI0DPHCBg7GNBiQgY6kst1bJKPdWX2elbwGjlX4cCUiflKo6qUCuig3HsdPA6vXGdLHQ3mowkqleWCodHVNHHGm0Rsu+e4r8VopP2YkLyFoVenP+e/8ARZ9QDks2AZeKBrinO2OezfA= ARC-Authentication-Results: i=1; mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1569340082556632.1632703864767; Tue, 24 Sep 2019 08:48:02 -0700 (PDT) Received: from localhost ([::1]:47398 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iCn2l-0006yf-I8 for importer@patchew.org; Tue, 24 Sep 2019 11:48:01 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:42802) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iCm6o-0007ro-N4 for qemu-devel@nongnu.org; Tue, 24 Sep 2019 10:48:07 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iCm6n-0003wD-6s for qemu-devel@nongnu.org; Tue, 24 Sep 2019 10:48:06 -0400 Received: from mx1.redhat.com ([209.132.183.28]:45780) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1iCm6m-0003ve-Uw; Tue, 24 Sep 2019 10:48:05 -0400 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.phx2.redhat.com [10.5.11.22]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 41CDE7DD11; Tue, 24 Sep 2019 14:48:04 +0000 (UTC) Received: from dell-r430-03.lab.eng.brq.redhat.com (dell-r430-03.lab.eng.brq.redhat.com [10.37.153.18]) by smtp.corp.redhat.com (Postfix) with ESMTP id 953AF10013D9; Tue, 24 Sep 2019 14:48:02 +0000 (UTC) From: Igor Mammedov To: qemu-devel@nongnu.org Subject: [PATCH v7 1/4] kvm: extract kvm_log_clear_one_slot Date: Tue, 24 Sep 2019 10:47:48 -0400 Message-Id: <20190924144751.24149-2-imammedo@redhat.com> In-Reply-To: <20190924144751.24149-1-imammedo@redhat.com> References: <20190924144751.24149-1-imammedo@redhat.com> X-Scanned-By: MIMEDefang 2.84 on 10.5.11.22 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.26]); Tue, 24 Sep 2019 14:48:04 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 209.132.183.28 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: thuth@redhat.com, david@redhat.com, cohuck@redhat.com, peterx@redhat.com, borntraeger@de.ibm.com, qemu-s390x@nongnu.org, pbonzini@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: Paolo Bonzini We may need to clear the dirty bitmap for more than one KVM memslot. First do some code movement with no semantic change. Signed-off-by: Paolo Bonzini Signed-off-by: Igor Mammedov Reviewed-by: Peter Xu Tested-by: Christian Borntraeger manual_dirty_log_protect) { - /* No need to do explicit clear */ - return 0; - } - - start =3D section->offset_within_address_space; - size =3D int128_get64(section->size); - - if (!size) { - /* Nothing more we can do... */ - return 0; - } - - kvm_slots_lock(kml); - - /* Find any possible slot that covers the section */ - for (i =3D 0; i < s->nr_slots; i++) { - mem =3D &kml->slots[i]; - if (mem->start_addr <=3D start && - start + size <=3D mem->start_addr + mem->memory_size) { - break; - } - } - - /* - * We should always find one memslot until this point, otherwise - * there could be something wrong from the upper layer - */ - assert(mem && i !=3D s->nr_slots); + int ret; =20 /* * We need to extend either the start or the size or both to @@ -694,7 +652,7 @@ static int kvm_physical_log_clear(KVMMemoryListener *km= l, /* It should never overflow. If it happens, say something */ assert(bmap_npages <=3D UINT32_MAX); d.num_pages =3D bmap_npages; - d.slot =3D mem->slot | (kml->as_id << 16); + d.slot =3D mem->slot | (as_id << 16); =20 if (kvm_vm_ioctl(s, KVM_CLEAR_DIRTY_LOG, &d) =3D=3D -1) { ret =3D -errno; @@ -717,6 +675,58 @@ static int kvm_physical_log_clear(KVMMemoryListener *k= ml, size / psize); /* This handles the NULL case well */ g_free(bmap_clear); + return ret; +} + + +/** + * kvm_physical_log_clear - Clear the kernel's dirty bitmap for range + * + * NOTE: this will be a no-op if we haven't enabled manual dirty log + * protection in the host kernel because in that case this operation + * will be done within log_sync(). + * + * @kml: the kvm memory listener + * @section: the memory range to clear dirty bitmap + */ +static int kvm_physical_log_clear(KVMMemoryListener *kml, + MemoryRegionSection *section) +{ + KVMState *s =3D kvm_state; + uint64_t start, size; + KVMSlot *mem =3D NULL; + int ret, i; + + if (!s->manual_dirty_log_protect) { + /* No need to do explicit clear */ + return 0; + } + + start =3D section->offset_within_address_space; + size =3D int128_get64(section->size); + + if (!size) { + /* Nothing more we can do... */ + return 0; + } + + kvm_slots_lock(kml); + + /* Find any possible slot that covers the section */ + for (i =3D 0; i < s->nr_slots; i++) { + mem =3D &kml->slots[i]; + if (mem->start_addr <=3D start && + start + size <=3D mem->start_addr + mem->memory_size) { + break; + } + } + + /* + * We should always find one memslot until this point, otherwise + * there could be something wrong from the upper layer + */ + assert(mem && i !=3D s->nr_slots); + ret =3D kvm_log_clear_one_slot(mem, kml->as_id, start, size); =20 kvm_slots_unlock(kml); =20 --=20 2.18.1 From nobody Sun Apr 28 23:35:52 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1569336656; cv=none; d=zoho.com; s=zohoarc; b=hUPzX5s61JxRHgks8i4RLOZJaIDAfGcCJIcKtX9dIJyY/9LQ4bRMqYz159kbXiijg/8eLUo8hczUvhEDnRfW3xChQ+uInlT0gQjyQjzjcncntuEvqwYZnfP4gvEu88I/XAlpzeNkv3hP3swYT04zCHJUo821l4QTbL2oiFpIEW8= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1569336656; h=Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:Message-ID:References:Sender:Subject:To:ARC-Authentication-Results; bh=IJnsI5Sgv0LQU+doa312yc4jFLyzkm2XTPMWwDp1dPg=; b=QwKLgidLyib1y1ooauwiuI0Ittcvs0HrN79SFS1Dd4Y7oeD51f0DEP+f565wmS9WbgeAvCk+4noPpRu7h2m0l88WOYiVWkrUf8xvouSFTQ25+BKCeMRpbreIHD0yONAsLaHvJcItrHQ4C7DXGfaLswAz/g9QqZP+jyL8taR+oa4= ARC-Authentication-Results: i=1; mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 156933665699536.42722381751287; Tue, 24 Sep 2019 07:50:56 -0700 (PDT) Received: from localhost ([::1]:46580 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iCm9X-0001mY-67 for importer@patchew.org; Tue, 24 Sep 2019 10:50:55 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:42849) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iCm6u-0007ww-9q for qemu-devel@nongnu.org; Tue, 24 Sep 2019 10:48:13 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iCm6t-0003zk-0I for qemu-devel@nongnu.org; Tue, 24 Sep 2019 10:48:12 -0400 Received: from mx1.redhat.com ([209.132.183.28]:41050) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1iCm6s-0003zM-NM; Tue, 24 Sep 2019 10:48:10 -0400 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.phx2.redhat.com [10.5.11.22]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 06A37302246D; Tue, 24 Sep 2019 14:48:10 +0000 (UTC) Received: from dell-r430-03.lab.eng.brq.redhat.com (dell-r430-03.lab.eng.brq.redhat.com [10.37.153.18]) by smtp.corp.redhat.com (Postfix) with ESMTP id 8474910013D9; Tue, 24 Sep 2019 14:48:04 +0000 (UTC) From: Igor Mammedov To: qemu-devel@nongnu.org Subject: [PATCH v7 2/4] kvm: clear dirty bitmaps from all overlapping memslots Date: Tue, 24 Sep 2019 10:47:49 -0400 Message-Id: <20190924144751.24149-3-imammedo@redhat.com> In-Reply-To: <20190924144751.24149-1-imammedo@redhat.com> References: <20190924144751.24149-1-imammedo@redhat.com> X-Scanned-By: MIMEDefang 2.84 on 10.5.11.22 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.49]); Tue, 24 Sep 2019 14:48:10 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 209.132.183.28 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: thuth@redhat.com, david@redhat.com, cohuck@redhat.com, peterx@redhat.com, borntraeger@de.ibm.com, qemu-s390x@nongnu.org, pbonzini@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: Paolo Bonzini Currently MemoryRegionSection has 1:1 mapping to KVMSlot. However next patch will allow splitting MemoryRegionSection into several KVMSlot-s, make sure that kvm_physical_log_slot_clear() is able to handle such 1:N mapping. Signed-off-by: Paolo Bonzini Signed-off-by: Igor Mammedov Reviewed-by: Peter Xu Tested-by: Christian Borntraeger start_addr) & KVM_CLEAR_LOG_MASK; - start_delta =3D start - mem->start_addr - bmap_start; + bmap_start =3D start & KVM_CLEAR_LOG_MASK; + start_delta =3D start - bmap_start; bmap_start /=3D psize; =20 /* @@ -693,8 +693,8 @@ static int kvm_physical_log_clear(KVMMemoryListener *km= l, MemoryRegionSection *section) { KVMState *s =3D kvm_state; - uint64_t start, size; - KVMSlot *mem =3D NULL; + uint64_t start, size, offset, count; + KVMSlot *mem; int ret, i; =20 if (!s->manual_dirty_log_protect) { @@ -712,22 +712,30 @@ static int kvm_physical_log_clear(KVMMemoryListener *= kml, =20 kvm_slots_lock(kml); =20 - /* Find any possible slot that covers the section */ for (i =3D 0; i < s->nr_slots; i++) { mem =3D &kml->slots[i]; - if (mem->start_addr <=3D start && - start + size <=3D mem->start_addr + mem->memory_size) { + /* Discard slots that are empty or do not overlap the section */ + if (!mem->memory_size || + mem->start_addr > start + size - 1 || + start > mem->start_addr + mem->memory_size - 1) { + continue; + } + + if (start >=3D mem->start_addr) { + /* The slot starts before section or is aligned to it. */ + offset =3D start - mem->start_addr; + count =3D MIN(mem->memory_size - offset, size); + } else { + /* The slot starts after section. */ + offset =3D 0; + count =3D MIN(mem->memory_size, size - (mem->start_addr - star= t)); + } + ret =3D kvm_log_clear_one_slot(mem, kml->as_id, offset, count); + if (ret < 0) { break; } } =20 - /* - * We should always find one memslot until this point, otherwise - * there could be something wrong from the upper layer - */ - assert(mem && i !=3D s->nr_slots); - ret =3D kvm_log_clear_one_slot(mem, kml->as_id, start, size); - kvm_slots_unlock(kml); =20 return ret; --=20 2.18.1 From nobody Sun Apr 28 23:35:52 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1569340305; cv=none; d=zoho.com; s=zohoarc; b=Uv92zudHVawJcOmcldtZBaf7zN0toPtlVMpft4iOMeHi07Gdawra4wrLb/ryoLxRJQJlr6ZJZ9vdLkiz6t/UGIwACkme+7gJr7qUM1LLZYBCK4UBIGctBDnKXJAo6N6iq9PgZxJ6dU1tnOipqvbvnU0Sp4hWXZ7JP1Jfjh/weng= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1569340305; h=Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:Message-ID:References:Sender:Subject:To:ARC-Authentication-Results; bh=o+KwfuAEmZRN9rPDUhYy6+J2BmDzDuwpIyKcMr/lxeY=; b=EjWQPDT6dujKhnxybJHRkcfjR7BE0k13LcGKOGqHosr7B+r4VEpQL2BJMnhgfThUGSW7a/AUFnGwMce2teSHKPF+XdnL0mzZKWa3m5YUUVyoPMeG0MvT39hB6sxPwaLXNRpGJMAqGKE5ec5f+SeSg1D6q/yc39sXKrko7EQRcEM= ARC-Authentication-Results: i=1; mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1569340305850618.6723777809415; Tue, 24 Sep 2019 08:51:45 -0700 (PDT) Received: from localhost ([::1]:47482 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iCn6O-0003Xu-4Y for importer@patchew.org; Tue, 24 Sep 2019 11:51:44 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:42867) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iCm6w-0007zu-GT for qemu-devel@nongnu.org; Tue, 24 Sep 2019 10:48:16 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iCm6u-000420-R0 for qemu-devel@nongnu.org; Tue, 24 Sep 2019 10:48:14 -0400 Received: from mx1.redhat.com ([209.132.183.28]:37072) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1iCm6u-00041K-JU; Tue, 24 Sep 2019 10:48:12 -0400 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.phx2.redhat.com [10.5.11.22]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id E4BC33CA03; Tue, 24 Sep 2019 14:48:11 +0000 (UTC) Received: from dell-r430-03.lab.eng.brq.redhat.com (dell-r430-03.lab.eng.brq.redhat.com [10.37.153.18]) by smtp.corp.redhat.com (Postfix) with ESMTP id 4B92B10013D9; Tue, 24 Sep 2019 14:48:10 +0000 (UTC) From: Igor Mammedov To: qemu-devel@nongnu.org Subject: [PATCH v7 3/4] kvm: split too big memory section on several memslots Date: Tue, 24 Sep 2019 10:47:50 -0400 Message-Id: <20190924144751.24149-4-imammedo@redhat.com> In-Reply-To: <20190924144751.24149-1-imammedo@redhat.com> References: <20190924144751.24149-1-imammedo@redhat.com> X-Scanned-By: MIMEDefang 2.84 on 10.5.11.22 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.39]); Tue, 24 Sep 2019 14:48:11 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 209.132.183.28 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: thuth@redhat.com, david@redhat.com, cohuck@redhat.com, peterx@redhat.com, borntraeger@de.ibm.com, qemu-s390x@nongnu.org, pbonzini@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Max memslot size supported by kvm on s390 is 8Tb, move logic of splitting RAM in chunks upto 8T to KVM code. This way it will hide KVM specific restrictions in KVM code and won't affect board level design decisions. Which would allow us to avoid misusing memory_region_allocate_system_memory() API and eventually use a single hostmem backend for guest RAM. Signed-off-by: Igor Mammedov Reviewed-by: Peter Xu Tested-by: Christian Borntraeger ) v4: * fix compilation issue (Christian Borntraeger ) * advance HVA along with GPA in kvm_set_phys_mem() (Christian Borntraeger ) patch prepares only KVM side for switching to single RAM memory region another patch will take care of dropping manual RAM partitioning in s390 code. --- include/sysemu/kvm_int.h | 1 + accel/kvm/kvm-all.c | 124 +++++++++++++++++++++++++-------------- 2 files changed, 81 insertions(+), 44 deletions(-) diff --git a/include/sysemu/kvm_int.h b/include/sysemu/kvm_int.h index 72b2d1b3ae..ac2d1f8b56 100644 --- a/include/sysemu/kvm_int.h +++ b/include/sysemu/kvm_int.h @@ -41,4 +41,5 @@ typedef struct KVMMemoryListener { void kvm_memory_listener_register(KVMState *s, KVMMemoryListener *kml, AddressSpace *as, int as_id); =20 +void kvm_set_max_memslot_size(hwaddr max_slot_size); #endif diff --git a/accel/kvm/kvm-all.c b/accel/kvm/kvm-all.c index 315a91557f..f3848e7e75 100644 --- a/accel/kvm/kvm-all.c +++ b/accel/kvm/kvm-all.c @@ -140,6 +140,7 @@ bool kvm_direct_msi_allowed; bool kvm_ioeventfd_any_length_allowed; bool kvm_msi_use_devid; static bool kvm_immediate_exit; +static hwaddr kvm_max_slot_size =3D ~0; =20 static const KVMCapabilityInfo kvm_required_capabilites[] =3D { KVM_CAP_INFO(USER_MEMORY), @@ -437,7 +438,7 @@ static int kvm_slot_update_flags(KVMMemoryListener *kml= , KVMSlot *mem, static int kvm_section_update_flags(KVMMemoryListener *kml, MemoryRegionSection *section) { - hwaddr start_addr, size; + hwaddr start_addr, size, slot_size; KVMSlot *mem; int ret =3D 0; =20 @@ -448,13 +449,18 @@ static int kvm_section_update_flags(KVMMemoryListener= *kml, =20 kvm_slots_lock(kml); =20 - mem =3D kvm_lookup_matching_slot(kml, start_addr, size); - if (!mem) { - /* We don't have a slot if we want to trap every access. */ - goto out; - } + while (size && !ret) { + slot_size =3D MIN(kvm_max_slot_size, size); + mem =3D kvm_lookup_matching_slot(kml, start_addr, slot_size); + if (!mem) { + /* We don't have a slot if we want to trap every access. */ + goto out; + } =20 - ret =3D kvm_slot_update_flags(kml, mem, section->mr); + ret =3D kvm_slot_update_flags(kml, mem, section->mr); + start_addr +=3D slot_size; + size -=3D slot_size; + } =20 out: kvm_slots_unlock(kml); @@ -527,11 +533,15 @@ static int kvm_physical_sync_dirty_bitmap(KVMMemoryLi= stener *kml, struct kvm_dirty_log d =3D {}; KVMSlot *mem; hwaddr start_addr, size; + hwaddr slot_size, slot_offset =3D 0; int ret =3D 0; =20 size =3D kvm_align_section(section, &start_addr); - if (size) { - mem =3D kvm_lookup_matching_slot(kml, start_addr, size); + while (size) { + MemoryRegionSection subsection =3D *section; + + slot_size =3D MIN(kvm_max_slot_size, size); + mem =3D kvm_lookup_matching_slot(kml, start_addr, slot_size); if (!mem) { /* We don't have a slot if we want to trap every access. */ goto out; @@ -549,11 +559,11 @@ static int kvm_physical_sync_dirty_bitmap(KVMMemoryLi= stener *kml, * So for now, let's align to 64 instead of HOST_LONG_BITS here, in * a hope that sizeof(long) won't become >8 any time soon. */ - size =3D ALIGN(((mem->memory_size) >> TARGET_PAGE_BITS), - /*HOST_LONG_BITS*/ 64) / 8; if (!mem->dirty_bmap) { + hwaddr bitmap_size =3D ALIGN(((mem->memory_size) >> TARGET_PAG= E_BITS), + /*HOST_LONG_BITS*/ 64) / 8; /* Allocate on the first log_sync, once and for all */ - mem->dirty_bmap =3D g_malloc0(size); + mem->dirty_bmap =3D g_malloc0(bitmap_size); } =20 d.dirty_bitmap =3D mem->dirty_bmap; @@ -564,7 +574,13 @@ static int kvm_physical_sync_dirty_bitmap(KVMMemoryLis= tener *kml, goto out; } =20 - kvm_get_dirty_pages_log_range(section, d.dirty_bitmap); + subsection.offset_within_region +=3D slot_offset; + subsection.size =3D int128_make64(slot_size); + kvm_get_dirty_pages_log_range(&subsection, d.dirty_bitmap); + + slot_offset +=3D slot_size; + start_addr +=3D slot_size; + size -=3D slot_size; } out: return ret; @@ -971,6 +987,14 @@ kvm_check_extension_list(KVMState *s, const KVMCapabil= ityInfo *list) return NULL; } =20 +void kvm_set_max_memslot_size(hwaddr max_slot_size) +{ + g_assert( + ROUND_UP(max_slot_size, qemu_real_host_page_size) =3D=3D max_slot_= size + ); + kvm_max_slot_size =3D max_slot_size; +} + static void kvm_set_phys_mem(KVMMemoryListener *kml, MemoryRegionSection *section, bool add) { @@ -978,7 +1002,7 @@ static void kvm_set_phys_mem(KVMMemoryListener *kml, int err; MemoryRegion *mr =3D section->mr; bool writeable =3D !mr->readonly && !mr->rom_device; - hwaddr start_addr, size; + hwaddr start_addr, size, slot_size; void *ram; =20 if (!memory_region_is_ram(mr)) { @@ -1003,41 +1027,52 @@ static void kvm_set_phys_mem(KVMMemoryListener *kml, kvm_slots_lock(kml); =20 if (!add) { - mem =3D kvm_lookup_matching_slot(kml, start_addr, size); - if (!mem) { - goto out; - } - if (mem->flags & KVM_MEM_LOG_DIRTY_PAGES) { - kvm_physical_sync_dirty_bitmap(kml, section); - } + do { + slot_size =3D MIN(kvm_max_slot_size, size); + mem =3D kvm_lookup_matching_slot(kml, start_addr, slot_size); + if (!mem) { + goto out; + } + if (mem->flags & KVM_MEM_LOG_DIRTY_PAGES) { + kvm_physical_sync_dirty_bitmap(kml, section); + } =20 - /* unregister the slot */ - g_free(mem->dirty_bmap); - mem->dirty_bmap =3D NULL; - mem->memory_size =3D 0; - mem->flags =3D 0; - err =3D kvm_set_user_memory_region(kml, mem, false); - if (err) { - fprintf(stderr, "%s: error unregistering slot: %s\n", - __func__, strerror(-err)); - abort(); - } + /* unregister the slot */ + g_free(mem->dirty_bmap); + mem->dirty_bmap =3D NULL; + mem->memory_size =3D 0; + mem->flags =3D 0; + err =3D kvm_set_user_memory_region(kml, mem, false); + if (err) { + fprintf(stderr, "%s: error unregistering slot: %s\n", + __func__, strerror(-err)); + abort(); + } + start_addr +=3D slot_size; + size -=3D slot_size; + } while (size); goto out; } =20 /* register the new slot */ - mem =3D kvm_alloc_slot(kml); - mem->memory_size =3D size; - mem->start_addr =3D start_addr; - mem->ram =3D ram; - mem->flags =3D kvm_mem_flags(mr); - - err =3D kvm_set_user_memory_region(kml, mem, true); - if (err) { - fprintf(stderr, "%s: error registering slot: %s\n", __func__, - strerror(-err)); - abort(); - } + do { + slot_size =3D MIN(kvm_max_slot_size, size); + mem =3D kvm_alloc_slot(kml); + mem->memory_size =3D slot_size; + mem->start_addr =3D start_addr; + mem->ram =3D ram; + mem->flags =3D kvm_mem_flags(mr); + + err =3D kvm_set_user_memory_region(kml, mem, true); + if (err) { + fprintf(stderr, "%s: error registering slot: %s\n", __func__, + strerror(-err)); + abort(); + } + start_addr +=3D slot_size; + ram +=3D slot_size; + size -=3D slot_size; + } while (size); =20 out: kvm_slots_unlock(kml); @@ -2877,6 +2912,7 @@ static bool kvm_accel_has_memory(MachineState *ms, Ad= dressSpace *as, =20 for (i =3D 0; i < kvm->nr_as; ++i) { if (kvm->as[i].as =3D=3D as && kvm->as[i].ml) { + size =3D MIN(kvm_max_slot_size, size); return NULL !=3D kvm_lookup_matching_slot(kvm->as[i].ml, start_addr, size); } --=20 2.18.1 From nobody Sun Apr 28 23:35:52 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1569337444; cv=none; d=zoho.com; s=zohoarc; b=X5+DmZISI3TXgHAWpipnIdCt+Tq9CFhsG7C3aBIIkvfrOAuJPR7AxADRQJfFK+QuA6i6jPGbbmyq4FVlLdjs5zuEqJ1HYHwHFJjLMwbtStBKSzC3d4TNGjIKl3E+xFlNnN47c5nItMauyHsHDCpNVn6CanDvyji0CjwX6zstEsE= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1569337444; h=Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:Message-ID:References:Sender:Subject:To:ARC-Authentication-Results; bh=cyqBaPMbFLcZgZieq0iyJc1kFh3HeiOKWNjen+ZMS5k=; b=NLd7X6z5f+IRljbppdk25om3P89FDLlIFDi54vk8o2BrSfK2aurYrwqKUDuWquj14YQFYBR3/961W8GeigLKeumKIzd5kvR/0M8hvrTRWheebM0A+7qXV5YBkujtU/VnruoLOhZ/LAcimqWn7CtlKbB8IxZCr4D+7LSXtzGnVeQ= ARC-Authentication-Results: i=1; mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1569337444884106.73405318879361; Tue, 24 Sep 2019 08:04:04 -0700 (PDT) Received: from localhost ([::1]:46734 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iCmMA-00061k-89 for importer@patchew.org; Tue, 24 Sep 2019 11:03:58 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:42885) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iCm6y-00081y-AK for qemu-devel@nongnu.org; Tue, 24 Sep 2019 10:48:17 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iCm6w-000439-RT for qemu-devel@nongnu.org; Tue, 24 Sep 2019 10:48:16 -0400 Received: from mx1.redhat.com ([209.132.183.28]:46676) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1iCm6w-00042e-JP; Tue, 24 Sep 2019 10:48:14 -0400 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.phx2.redhat.com [10.5.11.22]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id E3E1B99C42; Tue, 24 Sep 2019 14:48:13 +0000 (UTC) Received: from dell-r430-03.lab.eng.brq.redhat.com (dell-r430-03.lab.eng.brq.redhat.com [10.37.153.18]) by smtp.corp.redhat.com (Postfix) with ESMTP id 38DAF10013D9; Tue, 24 Sep 2019 14:48:12 +0000 (UTC) From: Igor Mammedov To: qemu-devel@nongnu.org Subject: [PATCH v7 4/4] s390: do not call memory_region_allocate_system_memory() multiple times Date: Tue, 24 Sep 2019 10:47:51 -0400 Message-Id: <20190924144751.24149-5-imammedo@redhat.com> In-Reply-To: <20190924144751.24149-1-imammedo@redhat.com> References: <20190924144751.24149-1-imammedo@redhat.com> X-Scanned-By: MIMEDefang 2.84 on 10.5.11.22 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.38]); Tue, 24 Sep 2019 14:48:13 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 209.132.183.28 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: thuth@redhat.com, david@redhat.com, cohuck@redhat.com, peterx@redhat.com, borntraeger@de.ibm.com, qemu-s390x@nongnu.org, pbonzini@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" s390 was trying to solve limited KVM memslot size issue by abusing memory_region_allocate_system_memory(), which breaks API contract where the function might be called only once. Beside an invalid use of API, the approach also introduced migration issue, since RAM chunks for each KVM_SLOT_MAX_BYTES are transferred in migration stream as separate RAMBlocks. After discussion [1], it was agreed to break migration from older QEMU for guest with RAM >8Tb (as it was relatively new (since 2.12) and considered to be not actually used downstream). Migration should keep working for guests with less than 8TB and for more than 8TB with QEMU 4.2 and newer binary. In case user tries to migrate more than 8TB guest, between incompatible QEMU versions, migration should fail gracefully due to non-exiting RAMBlock ID or RAMBlock size mismatch. Taking in account above and that now KVM code is able to split too big MemorySection into several memslots, partially revert commit (bb223055b s390-ccw-virtio: allow for systems larger that 7.999TB) and use kvm_set_max_memslot_size() to set KVMSlot size to KVM_SLOT_MAX_BYTES. 1) [PATCH RFC v2 4/4] s390: do not call memory_region_allocate_system_memo= ry() multiple times Signed-off-by: Igor Mammedov Acked-by: Igor Mammedov Acked-by: Paolo Bonzini Acked-by: Peter Xu Tested-by: Christian Borntraeger ) v3: - drop migration compat code PS: I don't have access to a suitable system to test it with 8Tb split, so I've tested only hacked up KVM_SLOT_MAX_BYTES =3D 1Gb variant --- hw/s390x/s390-virtio-ccw.c | 30 +++--------------------------- target/s390x/kvm.c | 11 +++++++++++ 2 files changed, 14 insertions(+), 27 deletions(-) diff --git a/hw/s390x/s390-virtio-ccw.c b/hw/s390x/s390-virtio-ccw.c index 8bfb6684cb..18ad279a00 100644 --- a/hw/s390x/s390-virtio-ccw.c +++ b/hw/s390x/s390-virtio-ccw.c @@ -154,39 +154,15 @@ static void virtio_ccw_register_hcalls(void) virtio_ccw_hcall_early_printk); } =20 -/* - * KVM does only support memory slots up to KVM_MEM_MAX_NR_PAGES pages - * as the dirty bitmap must be managed by bitops that take an int as - * position indicator. If we have a guest beyond that we will split off - * new subregions. The split must happen on a segment boundary (1MB). - */ -#define KVM_MEM_MAX_NR_PAGES ((1ULL << 31) - 1) -#define SEG_MSK (~0xfffffULL) -#define KVM_SLOT_MAX_BYTES ((KVM_MEM_MAX_NR_PAGES * TARGET_PAGE_SIZE) & SE= G_MSK) static void s390_memory_init(ram_addr_t mem_size) { MemoryRegion *sysmem =3D get_system_memory(); - ram_addr_t chunk, offset =3D 0; - unsigned int number =3D 0; + MemoryRegion *ram =3D g_new(MemoryRegion, 1); Error *local_err =3D NULL; - gchar *name; =20 /* allocate RAM for core */ - name =3D g_strdup_printf("s390.ram"); - while (mem_size) { - MemoryRegion *ram =3D g_new(MemoryRegion, 1); - uint64_t size =3D mem_size; - - /* KVM does not allow memslots >=3D 8 TB */ - chunk =3D MIN(size, KVM_SLOT_MAX_BYTES); - memory_region_allocate_system_memory(ram, NULL, name, chunk); - memory_region_add_subregion(sysmem, offset, ram); - mem_size -=3D chunk; - offset +=3D chunk; - g_free(name); - name =3D g_strdup_printf("s390.ram.%u", ++number); - } - g_free(name); + memory_region_allocate_system_memory(ram, NULL, "s390.ram", mem_size); + memory_region_add_subregion(sysmem, 0, ram); =20 /* * Configure the maximum page size. As no memory devices were created diff --git a/target/s390x/kvm.c b/target/s390x/kvm.c index 97a662ad0e..54864c259c 100644 --- a/target/s390x/kvm.c +++ b/target/s390x/kvm.c @@ -28,6 +28,7 @@ #include "cpu.h" #include "internal.h" #include "kvm_s390x.h" +#include "sysemu/kvm_int.h" #include "qapi/error.h" #include "qemu/error-report.h" #include "qemu/timer.h" @@ -122,6 +123,15 @@ */ #define VCPU_IRQ_BUF_SIZE(max_cpus) (sizeof(struct kvm_s390_irq) * \ (max_cpus + NR_LOCAL_IRQS)) +/* + * KVM does only support memory slots up to KVM_MEM_MAX_NR_PAGES pages + * as the dirty bitmap must be managed by bitops that take an int as + * position indicator. If we have a guest beyond that we will split off + * new subregions. The split must happen on a segment boundary (1MB). + */ +#define KVM_MEM_MAX_NR_PAGES ((1ULL << 31) - 1) +#define SEG_MSK (~0xfffffULL) +#define KVM_SLOT_MAX_BYTES ((KVM_MEM_MAX_NR_PAGES * TARGET_PAGE_SIZE) & SE= G_MSK) =20 static CPUWatchpoint hw_watchpoint; /* @@ -355,6 +365,7 @@ int kvm_arch_init(MachineState *ms, KVMState *s) */ /* kvm_vm_enable_cap(s, KVM_CAP_S390_AIS, 0); */ =20 + kvm_set_max_memslot_size(KVM_SLOT_MAX_BYTES); return 0; } =20 --=20 2.18.1