From nobody Sun Nov 24 09:06:26 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1725991200; cv=none; d=zohomail.com; s=zohoarc; b=Xl6j9ENUZNMtmiFjVe38R6A7KKKAaP1pvXMepywl7VvyCcqT3QD5IdHHf74eyAun8svi9/mg4DSheCNmEn5unLoGANP876HGagWr5mE2UlFHPQzOPmn+mwgVVdFjH7iiESkt83WcegcweRjJoLaEH1Lvk2rigEnIsq1FQAfd+ak= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1725991200; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=76s1XbThi25DI/WfFTNV4gn3SVd304dQvKR/FcJRkBE=; b=TmYsLG0+mVcwRTsMXvrN2QRFzM/sdMOvX9q646EFD4ZCDs6Rsd2Fqd96O50ghZ9YRC3Jo8bJ8ZOFWRNewsHFqh6pPV9CBj9gU661KAoiREfObfXtKG8wuf72PlLYiLplnC+dQLG98Diz5/h2LclNtsiHEYPdYka94BHUjQmiwLI= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1725991200919273.5259613657364; Tue, 10 Sep 2024 11:00:00 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1so58s-00074T-OH; Tue, 10 Sep 2024 13:59:06 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1so58q-0006va-3i for qemu-devel@nongnu.org; Tue, 10 Sep 2024 13:59:04 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1so58o-0006R1-8Y for qemu-devel@nongnu.org; Tue, 10 Sep 2024 13:59:03 -0400 Received: from mx-prod-mc-05.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-140-ngDbFHKKPDeRlhZL7lg41A-1; Tue, 10 Sep 2024 13:58:58 -0400 Received: from mx-prod-int-04.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-04.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.40]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-05.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 4A4F31956048; Tue, 10 Sep 2024 17:58:57 +0000 (UTC) Received: from t14s.redhat.com (unknown [10.22.32.182]) by mx-prod-int-04.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 2B61B19560AD; Tue, 10 Sep 2024 17:58:52 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1725991141; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=76s1XbThi25DI/WfFTNV4gn3SVd304dQvKR/FcJRkBE=; b=Cp5nsTfPoIWT+xTk9bE06TUZctCGvMKJ73bEC1otg2yWt8PoIK/J68iDy8pwb5s5Mi0rYH DuU086HOoynEYJkdmFPTPtD2ZKni9CoFSh8hKLamUymDfm7Pja3rrKQ995hgnDp0LTu2YJ 1JI87NAm5cdDQYgFUlG1+A+4ZZWcGUk= X-MC-Unique: ngDbFHKKPDeRlhZL7lg41A-1 From: David Hildenbrand To: qemu-devel@nongnu.org Cc: qemu-s390x@nongnu.org, David Hildenbrand , Paolo Bonzini , Thomas Huth , Halil Pasic , Christian Borntraeger , Eric Farman , Richard Henderson , Ilya Leoshkevich , Janosch Frank , "Michael S. Tsirkin" , Cornelia Huck Subject: [PATCH v1 08/14] s390x/s390-stattrib-kvm: prepare memory devices and sparse memory layouts Date: Tue, 10 Sep 2024 19:58:03 +0200 Message-ID: <20240910175809.2135596-9-david@redhat.com> In-Reply-To: <20240910175809.2135596-1-david@redhat.com> References: <20240910175809.2135596-1-david@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.0 on 10.30.177.40 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=david@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -21 X-Spam_score: -2.2 X-Spam_bar: -- X-Spam_report: (-2.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.145, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1725991201736116600 Content-Type: text/plain; charset="utf-8" With memory devices, we will have storage attributes for memory that exceeds the initial ram size. Further, we can easily have memory holes, for which there (currently) are no storage attributes. In particular, with memory holes, KVM_S390_SET_CMMA_BITS will fail to set some storage attributes. So let's do it like we handle storage keys migration, relying on guest_phys_blocks_append(). However, in contrast to storage key migration, we will handle it on the migration destination. This is a preparation for virtio-mem support. Note that ever since the "early migration" feature was added (x-early-migration), the state of device blocks (plugged/unplugged) is migrated early such that guest_phys_blocks_append() will properly consider all currently plugged memory blocks and skip any unplugged ones. In the future, we should try getting rid of the large temporary buffer and also not send any attributes for any memory holes, just so they get ignored on the destination. Signed-off-by: David Hildenbrand --- hw/s390x/s390-stattrib-kvm.c | 63 +++++++++++++++++++++++------------- 1 file changed, 40 insertions(+), 23 deletions(-) diff --git a/hw/s390x/s390-stattrib-kvm.c b/hw/s390x/s390-stattrib-kvm.c index eeaa811098..1f1bd507b5 100644 --- a/hw/s390x/s390-stattrib-kvm.c +++ b/hw/s390x/s390-stattrib-kvm.c @@ -15,6 +15,7 @@ #include "hw/s390x/storage-attributes.h" #include "qemu/error-report.h" #include "sysemu/kvm.h" +#include "sysemu/memory_mapping.h" #include "exec/ram_addr.h" #include "kvm/kvm_s390x.h" #include "qapi/error.h" @@ -84,8 +85,7 @@ static int kvm_s390_stattrib_set_stattr(S390StAttribState= *sa, uint8_t *values) { KVMS390StAttribState *sas =3D KVM_S390_STATTRIB(sa); - MachineState *machine =3D MACHINE(qdev_get_machine()); - unsigned long max =3D machine->ram_size / TARGET_PAGE_SIZE; + unsigned long max =3D s390_get_memory_limit() / TARGET_PAGE_SIZE; =20 if (start_gfn + count > max) { error_report("Out of memory bounds when setting storage attributes= "); @@ -103,39 +103,56 @@ static int kvm_s390_stattrib_set_stattr(S390StAttribS= tate *sa, static void kvm_s390_stattrib_synchronize(S390StAttribState *sa) { KVMS390StAttribState *sas =3D KVM_S390_STATTRIB(sa); - MachineState *machine =3D MACHINE(qdev_get_machine()); - unsigned long max =3D machine->ram_size / TARGET_PAGE_SIZE; - /* We do not need to reach the maximum buffer size allowed */ - unsigned long cx, len =3D KVM_S390_SKEYS_MAX / 2; + unsigned long max =3D s390_get_memory_limit() / TARGET_PAGE_SIZE; + unsigned long start_gfn, end_gfn, pages; + GuestPhysBlockList guest_phys_blocks; + GuestPhysBlock *block; int r; struct kvm_s390_cmma_log clog =3D { .flags =3D 0, .mask =3D ~0ULL, }; =20 - if (sas->incoming_buffer) { - for (cx =3D 0; cx + len <=3D max; cx +=3D len) { - clog.start_gfn =3D cx; - clog.count =3D len; - clog.values =3D (uint64_t)(sas->incoming_buffer + cx); - r =3D kvm_vm_ioctl(kvm_state, KVM_S390_SET_CMMA_BITS, &clog); - if (r) { - error_report("KVM_S390_SET_CMMA_BITS failed: %s", strerror= (-r)); - return; - } - } - if (cx < max) { - clog.start_gfn =3D cx; - clog.count =3D max - cx; - clog.values =3D (uint64_t)(sas->incoming_buffer + cx); + if (!sas->incoming_buffer) { + return; + } + guest_phys_blocks_init(&guest_phys_blocks); + guest_phys_blocks_append(&guest_phys_blocks); + + QTAILQ_FOREACH(block, &guest_phys_blocks.head, next) { + assert(QEMU_IS_ALIGNED(block->target_start, TARGET_PAGE_SIZE)); + assert(QEMU_IS_ALIGNED(block->target_end, TARGET_PAGE_SIZE)); + + start_gfn =3D block->target_start / TARGET_PAGE_SIZE; + end_gfn =3D block->target_end / TARGET_PAGE_SIZE; + + while (start_gfn < end_gfn) { + /* Don't exceed the maximum buffer size. */ + pages =3D MIN(end_gfn - start_gfn, KVM_S390_SKEYS_MAX / 2); + + /* + * If we ever get guest physical memory beyond the configured + * memory limit, something went very wrong. + */ + assert(start_gfn + pages <=3D max); + + clog.start_gfn =3D start_gfn; + clog.count =3D pages; + clog.values =3D (uint64_t)(sas->incoming_buffer + start_gfn); r =3D kvm_vm_ioctl(kvm_state, KVM_S390_SET_CMMA_BITS, &clog); if (r) { error_report("KVM_S390_SET_CMMA_BITS failed: %s", strerror= (-r)); + goto out; } + + start_gfn +=3D pages; } - g_free(sas->incoming_buffer); - sas->incoming_buffer =3D NULL; } + +out: + guest_phys_blocks_free(&guest_phys_blocks); + g_free(sas->incoming_buffer); + sas->incoming_buffer =3D NULL; } =20 static int kvm_s390_stattrib_set_migrationmode(S390StAttribState *sa, bool= val, --=20 2.46.0