From nobody Tue Dec 24 12:56:07 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1734809079; cv=none; d=zohomail.com; s=zohoarc; b=Jt1qNUdBIlbIFpZgnnfPwx/aDi3bA5I+Ye0qyZzU45M5e4mlVDpvAHLJtT+Fym30BdrBLI+BX5QuKtZ9WK3iuWnqLTts1W10duPNaYILLm8LSobivP8k2Gsd/jJJt3gbBYwOX61Iyv1LQG0yeye/SGh/OCHW367aNO7UDaN8QR8= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1734809079; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=TyHETmSFfRkani9FLrh+iELen8TJf3jh3hXnkuxCM7M=; b=MVzBRVzPfayoCpITpIqvd9CK8GbeP4BoqkZY8hkEZGHRffuP33IIuvLJczidoCmVVDnk8aH39DQ7J9oqWbvFAJkvxxG+V2Xt70dsd7I6NODcu93JOLm0KhYtxqv7W0hXU5Nlzzp1CjdBzpuVDShDahmbTpDZ5nqmYBCkv9lCqRo= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1734809079667431.3459107650033; Sat, 21 Dec 2024 11:24:39 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1tP53l-0002R6-JT; Sat, 21 Dec 2024 14:22:45 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tP53k-0002Qo-9Q for qemu-devel@nongnu.org; Sat, 21 Dec 2024 14:22:44 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tP53i-0000Kl-M6 for qemu-devel@nongnu.org; Sat, 21 Dec 2024 14:22:44 -0500 Received: from mail-wr1-f70.google.com (mail-wr1-f70.google.com [209.85.221.70]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-534-0QYQxfZWMh20j5WO0MSz0A-1; Sat, 21 Dec 2024 14:22:38 -0500 Received: by mail-wr1-f70.google.com with SMTP id ffacd0b85a97d-385df115288so1411046f8f.2 for ; Sat, 21 Dec 2024 11:22:38 -0800 (PST) Received: from localhost (p200300cbc7137800820d0e9f08ce52bd.dip0.t-ipconnect.de. [2003:cb:c713:7800:820d:e9f:8ce:52bd]) by smtp.gmail.com with UTF8SMTPSA id ffacd0b85a97d-38a1c8474a9sm7072492f8f.52.2024.12.21.11.22.34 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Sat, 21 Dec 2024 11:22:35 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1734808962; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=TyHETmSFfRkani9FLrh+iELen8TJf3jh3hXnkuxCM7M=; b=B/7mzEOsSvc5vMSmRHTPrVk00yPnvKixGjp3bW1gf1MjnDl8tsZ5URMj8f3jz3NmLPnw35 XxZx1i6mpSVCAkrWNfHncrBj/PCBLj2uQkPrX6rwfVghUp+ysbwqfEuAy3DUQ19B8m6iqL J4LMu+E9n1VgNZ/HKJX4dVtpIlQyoto= X-MC-Unique: 0QYQxfZWMh20j5WO0MSz0A-1 X-Mimecast-MFC-AGG-ID: 0QYQxfZWMh20j5WO0MSz0A X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1734808957; x=1735413757; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=TyHETmSFfRkani9FLrh+iELen8TJf3jh3hXnkuxCM7M=; b=hYAe+42ESsrhu/wRddFc9mH3l72qZdse9+6YN4KyrANGfRE+Sknd+kClG/0YYKLU3Q huTQY99Q+KhveooQtVQxKHCe6SU11GZRZYPHQNS7RXh2BIypMO78E7LRn4AoCdGs60s5 aSXm8YA9xyM+SHnpDAtbsvq8K3S6zyD+0X7KzX4TqDMKQSw8f870qn6RKUYMFksjb1cA Vwvn5C639eZAuEk6ceqeP1AMae66wYlb+lrPWf7BNlqh+9SypZzgF+26+sTjSdfzPT74 bxpSDVFImVAeHYNoUABKCnAWW59SbRoD8ywZkbe4EYIN3+T+R8/CHgsK7r+fGEAPBbdv iD4w== X-Gm-Message-State: AOJu0YxiNXMm58BQQ1zsnzParZQx7NBI+/OSgnXgYy6L/0h52pxAvEql Y74JVAWPTrgU99Wns9LYwD7tdg1pNUFBaxejoxzfXQJHwsnlM31tmiY+0R/9FYXNGLk0heE72VP WY3oFWUojH1XXViS4u06RfSI9eWNvFyL8Rk50P+IUo5UAH0OTdWk1SvvxiufYyqsvf9KGUWMspZ Pf72uMT1A0xSfAKxtTwh3NiMCz5cKVzYAn X-Gm-Gg: ASbGnctMcTc/9zAC0XhhCAnBS6KLOOyvcydNOxZAzmJeqxk7nq6P2CrOcAhDGtdLac+ RcB9Bfr4HmIfyxSz5bx078TLp/QI6+v6BtbMvrBJZeXy77FNa1zuJi7NFcRdE+uKrdf/z14mP2F j/soehlncPYdjV/RPLEnBlmh3nz7ktkA4FV5opf+3FdQKp8CmXTXT2gboDtbo22tl4aPtfYyg42 jD4XQuM0oT42YMXcgeIMdwfD7gD8Gper1sX2YlC8wmSoXmJcMomBHQAB5DOxrlbx544Daz7zPRU ChbF1gmVgdM7lPJmD2y/BQwe+IpbZ1jYIci2pyo= X-Received: by 2002:a05:6000:71b:b0:385:fab3:c56d with SMTP id ffacd0b85a97d-38a221685c4mr7121846f8f.0.1734808957402; Sat, 21 Dec 2024 11:22:37 -0800 (PST) X-Google-Smtp-Source: AGHT+IF1gaHU8eFWPAQqJ2ZbDNX9RSfGulkjNHaH8H6K5MjIqXuyLemJBQJ2UnH1RR6jGFO6+Swrrg== X-Received: by 2002:a05:6000:71b:b0:385:fab3:c56d with SMTP id ffacd0b85a97d-38a221685c4mr7121830f8f.0.1734808957060; Sat, 21 Dec 2024 11:22:37 -0800 (PST) From: David Hildenbrand To: qemu-devel@nongnu.org Cc: Halil Pasic , Christian Borntraeger , Eric Farman , Thomas Huth , Richard Henderson , "Michael S . Tsirkin" , David Hildenbrand Subject: [PULL v2 09/15] s390x/s390-stattrib-kvm: prepare for memory devices and sparse memory layouts Date: Sat, 21 Dec 2024 20:22:03 +0100 Message-ID: <20241221192209.3979595-10-david@redhat.com> X-Mailer: git-send-email 2.47.1 In-Reply-To: <20241221192209.3979595-1-david@redhat.com> References: <20241221192209.3979595-1-david@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=dhildenb@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -32 X-Spam_score: -3.3 X-Spam_bar: --- X-Spam_report: (-3.3 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-1.177, RCVD_IN_VALIDITY_CERTIFIED_BLOCKED=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1734809080230116600 Content-Type: text/plain; charset="utf-8" With memory devices, we will have storage attributes for memory that exceeds the initial ram size. Further, we can easily have memory holes, for which there (currently) are no storage attributes. In particular, with memory holes, KVM_S390_SET_CMMA_BITS will fail to set some storage attributes. So let's do it like we handle storage keys migration, relying on guest_phys_blocks_append(). However, in contrast to storage key migration, we will handle it on the migration destination. This is a preparation for virtio-mem support. Note that ever since the "early migration" feature was added (x-early-migration), the state of device blocks (plugged/unplugged) is migrated early such that guest_phys_blocks_append() will properly consider all currently plugged memory blocks and skip any unplugged ones. In the future, we should try getting rid of the large temporary buffer and also not send any attributes for any memory holes, just so they get ignored on the destination. Message-ID: <20241219144115.2820241-9-david@redhat.com> Acked-by: Michael S. Tsirkin Reviewed-by: Thomas Huth Signed-off-by: David Hildenbrand --- hw/s390x/s390-stattrib-kvm.c | 67 +++++++++++++++++++++++------------- 1 file changed, 43 insertions(+), 24 deletions(-) diff --git a/hw/s390x/s390-stattrib-kvm.c b/hw/s390x/s390-stattrib-kvm.c index eeaa811098..33ec91422a 100644 --- a/hw/s390x/s390-stattrib-kvm.c +++ b/hw/s390x/s390-stattrib-kvm.c @@ -10,11 +10,12 @@ */ =20 #include "qemu/osdep.h" -#include "hw/boards.h" +#include "hw/s390x/s390-virtio-ccw.h" #include "migration/qemu-file.h" #include "hw/s390x/storage-attributes.h" #include "qemu/error-report.h" #include "sysemu/kvm.h" +#include "sysemu/memory_mapping.h" #include "exec/ram_addr.h" #include "kvm/kvm_s390x.h" #include "qapi/error.h" @@ -84,8 +85,8 @@ static int kvm_s390_stattrib_set_stattr(S390StAttribState= *sa, uint8_t *values) { KVMS390StAttribState *sas =3D KVM_S390_STATTRIB(sa); - MachineState *machine =3D MACHINE(qdev_get_machine()); - unsigned long max =3D machine->ram_size / TARGET_PAGE_SIZE; + S390CcwMachineState *s390ms =3D S390_CCW_MACHINE(qdev_get_machine()); + unsigned long max =3D s390_get_memory_limit(s390ms) / TARGET_PAGE_SIZE; =20 if (start_gfn + count > max) { error_report("Out of memory bounds when setting storage attributes= "); @@ -103,39 +104,57 @@ static int kvm_s390_stattrib_set_stattr(S390StAttribS= tate *sa, static void kvm_s390_stattrib_synchronize(S390StAttribState *sa) { KVMS390StAttribState *sas =3D KVM_S390_STATTRIB(sa); - MachineState *machine =3D MACHINE(qdev_get_machine()); - unsigned long max =3D machine->ram_size / TARGET_PAGE_SIZE; - /* We do not need to reach the maximum buffer size allowed */ - unsigned long cx, len =3D KVM_S390_SKEYS_MAX / 2; + S390CcwMachineState *s390ms =3D S390_CCW_MACHINE(qdev_get_machine()); + unsigned long max =3D s390_get_memory_limit(s390ms) / TARGET_PAGE_SIZE; + unsigned long start_gfn, end_gfn, pages; + GuestPhysBlockList guest_phys_blocks; + GuestPhysBlock *block; int r; struct kvm_s390_cmma_log clog =3D { .flags =3D 0, .mask =3D ~0ULL, }; =20 - if (sas->incoming_buffer) { - for (cx =3D 0; cx + len <=3D max; cx +=3D len) { - clog.start_gfn =3D cx; - clog.count =3D len; - clog.values =3D (uint64_t)(sas->incoming_buffer + cx); - r =3D kvm_vm_ioctl(kvm_state, KVM_S390_SET_CMMA_BITS, &clog); - if (r) { - error_report("KVM_S390_SET_CMMA_BITS failed: %s", strerror= (-r)); - return; - } - } - if (cx < max) { - clog.start_gfn =3D cx; - clog.count =3D max - cx; - clog.values =3D (uint64_t)(sas->incoming_buffer + cx); + if (!sas->incoming_buffer) { + return; + } + guest_phys_blocks_init(&guest_phys_blocks); + guest_phys_blocks_append(&guest_phys_blocks); + + QTAILQ_FOREACH(block, &guest_phys_blocks.head, next) { + assert(QEMU_IS_ALIGNED(block->target_start, TARGET_PAGE_SIZE)); + assert(QEMU_IS_ALIGNED(block->target_end, TARGET_PAGE_SIZE)); + + start_gfn =3D block->target_start / TARGET_PAGE_SIZE; + end_gfn =3D block->target_end / TARGET_PAGE_SIZE; + + while (start_gfn < end_gfn) { + /* Don't exceed the maximum buffer size. */ + pages =3D MIN(end_gfn - start_gfn, KVM_S390_SKEYS_MAX / 2); + + /* + * If we ever get guest physical memory beyond the configured + * memory limit, something went very wrong. + */ + assert(start_gfn + pages <=3D max); + + clog.start_gfn =3D start_gfn; + clog.count =3D pages; + clog.values =3D (uint64_t)(sas->incoming_buffer + start_gfn); r =3D kvm_vm_ioctl(kvm_state, KVM_S390_SET_CMMA_BITS, &clog); if (r) { error_report("KVM_S390_SET_CMMA_BITS failed: %s", strerror= (-r)); + goto out; } + + start_gfn +=3D pages; } - g_free(sas->incoming_buffer); - sas->incoming_buffer =3D NULL; } + +out: + guest_phys_blocks_free(&guest_phys_blocks); + g_free(sas->incoming_buffer); + sas->incoming_buffer =3D NULL; } =20 static int kvm_s390_stattrib_set_migrationmode(S390StAttribState *sa, bool= val, --=20 2.47.1