From nobody Mon Feb 9 11:30:05 2026 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of redhat.com designates 170.10.133.124 as permitted sender) client-ip=170.10.133.124; envelope-from=libvir-list-bounces@redhat.com; helo=us-smtp-delivery-124.mimecast.com; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1631545017; cv=none; d=zohomail.com; s=zohoarc; b=HNGc+NAfT7cs8jEhj0ayB8mCKAX56Qu3Thb2PTzzM+qr8/LPmgv3kZGotuL8klQE7hpVWB1kR3rqgwmTBUCzHgTyFrIpCPW0M8uU8XtIcGIkutjD62HbzPiFze3kRrsxkSiRL82HJ2AlJ1f1qQAFWCN0OHP9FQP75c0C3QpzWmY= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1631545017; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=5Vr7xtZchMYJKtgaloUBkg9wcCFoqRpocUrkk7NGsFM=; b=ZeuCh7Fs3Iw2z83287+kD1hGjpf6Dd795StEvZTUTug7jg78t8QbUX60P9LrenfBBG/0YayupYLMa1bf6ud0Psd7LOtexDfIQZgtIpoeACanwRklrQAytptHWK7F2pE4FmYUYe7IBAduD2g8u6DMJropibHWS1XwRpfymxAUcpA= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by mx.zohomail.com with SMTPS id 1631545017818228.7739532965178; Mon, 13 Sep 2021 07:56:57 -0700 (PDT) Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-554-fek2c_rsOYGtIxP-y4PaiA-1; Mon, 13 Sep 2021 10:56:54 -0400 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id CDBBE100F968; Mon, 13 Sep 2021 14:56:48 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.21]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 6B2B35C23A; Mon, 13 Sep 2021 14:56:48 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id 3C2064EA30; Mon, 13 Sep 2021 14:56:48 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id 18DErevo021296 for ; Mon, 13 Sep 2021 10:53:40 -0400 Received: by smtp.corp.redhat.com (Postfix) id 668696B556; Mon, 13 Sep 2021 14:53:40 +0000 (UTC) Received: from maggie.redhat.com (unknown [10.43.2.93]) by smtp.corp.redhat.com (Postfix) with ESMTP id BB86E6A8F5; Mon, 13 Sep 2021 14:53:36 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1631545016; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post; bh=5Vr7xtZchMYJKtgaloUBkg9wcCFoqRpocUrkk7NGsFM=; b=JuYSA45RPALI6awDsXtmydlKGGAMMx16fWKg48LQ+M7tXG4ILjTZ22+PXR9A/blMO0hcP0 FkU42if3IeW05Kcyf9UVUG5+JlZ3+6R8tsbQ0gVr6+NZ2VQ49LGf/FhGwdwYG9Tz0pZ1Fm cA2+eFW1Xb2xheiSY4y87U5zG4mPW3M= X-MC-Unique: fek2c_rsOYGtIxP-y4PaiA-1 From: Michal Privoznik To: libvir-list@redhat.com Subject: [PATCH v5 16/16] kbase: Document virtio-mem Date: Mon, 13 Sep 2021 16:52:44 +0200 Message-Id: <551845aee7c2a5e5ebe34833ef33eaff510f4cc3.1631544341.git.mprivozn@redhat.com> In-Reply-To: References: MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-loop: libvir-list@redhat.com Cc: david@redhat.com X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=libvir-list-bounces@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1631545020023100005 Content-Type: text/plain; charset="utf-8" This commit adds new memorydevices.rst page which should serve all models of memory devices. Yet, I'm documenting virtio-mem quirks only. Signed-off-by: Michal Privoznik --- docs/kbase/index.rst | 4 + docs/kbase/memorydevices.rst | 150 +++++++++++++++++++++++++++++++++++ docs/kbase/meson.build | 1 + 3 files changed, 155 insertions(+) create mode 100644 docs/kbase/memorydevices.rst diff --git a/docs/kbase/index.rst b/docs/kbase/index.rst index 351ea2c93b..09b19ed1af 100644 --- a/docs/kbase/index.rst +++ b/docs/kbase/index.rst @@ -52,6 +52,10 @@ Usage `PCI topology <../pci-addresses.html>`__ Addressing schemes for PCI devices =20 +`Memory devices `__ + Memory devices and their use + + Internals / Debugging --------------------- =20 diff --git a/docs/kbase/memorydevices.rst b/docs/kbase/memorydevices.rst new file mode 100644 index 0000000000..7f58a4247b --- /dev/null +++ b/docs/kbase/memorydevices.rst @@ -0,0 +1,150 @@ +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D +Memory devices +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D + +.. contents:: + +Basics +=3D=3D=3D=3D=3D=3D + +Memory devices can be divided into two families: volatile and non-volatile. +The former is typical RAM memory: it's volatile and thus its contents does= n't +survive guest reboots or power cycles. The latter retains its contents acr= oss +reboots or power outages. + +In Libvirt, there are two models for volatile memory: + +* ``dimm`` model: + + :: + + + + 523264 + 0 + +
+ + +* ``virtio-mem`` model: + + :: + + + + 1048576 + 0 + 2048 + 524288 + +
+ + +Then there are two models for non-volatile memory: + +* ``nvdimm`` model: + + :: + + + + /tmp/nvdimm + + + 523264 + 0 + +
+ + +* ``virtio-pmem`` model: + + :: + + + + /tmp/virtio_pmem + + + 524288 + +
+ + + +Please note that (maybe somewhat surprisingly) virtio models go onto PCI b= us +instead of DIMM slots. + +Furthermore, DIMMs can have ```` element which configures backend= for +devices. For NVDIMMs the element is mandatory and reflects where the conte= nt +is saved. + +See `memory devices documentation <../formatdomain.html#elementsMemory>`_. + +``virtio-mem`` model +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D + +The ``virtio-mem`` model can be viewed as revised memory balloon. It offers +adding and removing memory (without the actual hotplug of the device). It +solves problems that memory balloon can't solve on its own and thus is more +flexible than DIMM + balloon solution. ``virtio-mem`` is NUMA aware, and t= hus +memory can be inflated/deflated only for a subset of guest NUMA nodes. Al= so, +it works with chunks that are either exposed to guest or reclaimed from it. + +See https://virtio-mem.gitlab.io/ + +Under the hood, ``virtio-mem`` device is split into chunks of equal size w= hich +are then exposed to the guest. Either all of them or only a portion depend= ing +on user's request. Therefore there are three important sizes for +``virtio-mem``. All are to be found under ```` element: + +#. The maximum size the device can ever offer, exposed under ```` +#. The size of a single block, exposed under ```` +#. The current size exposed to the guest, exposed under ```` + +For instance, in the following example the maximum size is 4GiB, the block= size +is 2MiB and only 1GiB should be exposed to the guest: + + :: + + + + 4194304 + 2048 + 1048576 + + + +Please note that ```` must be an integer multiple of ```` +size or zero (no blocks exposed to the guest) and has to be less or equal = to +```` (all blocks exposed to the guest). Furthermore, QEMU recommend= s the +```` size to be as big as a Transparent Huge Page (usually 2MiB). + +To change the size exposed to the guest, users should pass memory device X= ML +with nothing but ```` changed into the +``virDomainUpdateDeviceFlags()`` API. For user's convenience this can be d= one +via virsh too: + + :: + + # virsh update-memory-device $dom --requested-size 2GiB + +If there are two or more ```` devices then ``--alias`` shall be u= sed +to tell virsh which memory device should be updated. + +For running guests there is fourth size that can be found under ````: + + :: + + 2097152 + +The ```` reflects the current size used by the guest. In general= it +can differ from ````. Reasons include guest kernel missing +``virtio-mem`` module and thus being unable to take offered memory, or gue= st +kernel being unable to free memory. Since ```` only reports siz= e to +users, the element is never parsed. It is formatted only into live XML. + +Since changing ```` allocation requires cooperation with guest +kernel, requests for change are not instant. Therefore, libvirt emits +``VIR_DOMAIN_EVENT_ID_MEMORY_DEVICE_SIZE_CHANGE`` event whenever current +allocation changed. diff --git a/docs/kbase/meson.build b/docs/kbase/meson.build index 73dc11837f..af067db453 100644 --- a/docs/kbase/meson.build +++ b/docs/kbase/meson.build @@ -10,6 +10,7 @@ docs_kbase_files =3D [ 'locking-lockd', 'locking', 'locking-sanlock', + 'memorydevices', 'merging_disk_image_chains', 'migrationinternals', 'qemu-core-dump', --=20 2.32.0