From nobody Sun Feb 8 21:32:22 2026 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of redhat.com designates 216.205.24.124 as permitted sender) client-ip=216.205.24.124; envelope-from=libvir-list-bounces@redhat.com; helo=us-smtp-delivery-124.mimecast.com; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 216.205.24.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1632232445; cv=none; d=zohomail.com; s=zohoarc; b=U0gSPjTASGWOKAfgWbZOUW92vMjqFNylM4NX/9zehkaMnyK9av9Hg6GgLaQlxJpD/QBINR/3yy2cHWkjEvJ3CZ1i90lQqUpHEFxVkiOh/gUFw4/Sy5wjtKGSx7+FS9UKnUgEQLA9k9O4cfsZczLDZ7MiBLx+VNMmqybfe+h4dEU= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1632232445; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=5Vr7xtZchMYJKtgaloUBkg9wcCFoqRpocUrkk7NGsFM=; b=OkF7bvBM+3/6LROtkn+Ezz9lHzEgQpuLCLTpZWFwVNtDVlbiINJNcENiONvd0yOdRt7XT1m9CSAsM5upDEtwB22dtc8xb9Djq5y+ttPLJZKL75MaNnWEtD3jUYGILduwoiabookJ29XLVxxq73y1ySfBMCKrIXH/3Kx46mr/TAI= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 216.205.24.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [216.205.24.124]) by mx.zohomail.com with SMTPS id 1632232445172847.7611282122981; Tue, 21 Sep 2021 06:54:05 -0700 (PDT) Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-499-shPCPP4PMDCew2l_6hu4GA-1; Tue, 21 Sep 2021 09:53:36 -0400 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 89CAB835DE6; Tue, 21 Sep 2021 13:53:30 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.20]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 6BE2B60938; Tue, 21 Sep 2021 13:53:30 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id 3A6F21832DD5; Tue, 21 Sep 2021 13:53:30 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id 18LDrShn004841 for ; Tue, 21 Sep 2021 09:53:28 -0400 Received: by smtp.corp.redhat.com (Postfix) id 3FA3960877; Tue, 21 Sep 2021 13:53:28 +0000 (UTC) Received: from maggie.redhat.com (unknown [10.43.2.93]) by smtp.corp.redhat.com (Postfix) with ESMTP id 94AE85BAE9; Tue, 21 Sep 2021 13:53:27 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1632232443; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post; bh=5Vr7xtZchMYJKtgaloUBkg9wcCFoqRpocUrkk7NGsFM=; b=Nt/2GkUjVoL1pmGjj8lPrU1VwutrnrFI7gi4jKENUW0lQz1dgF3abQ6AhXwjhWsweETHjs HoJ7AtsgS2NMHjt17Sf8YVxEW6muxY/MO+jIQSefl9anaO3qcjUsiDoFNYnrnZ1RrF7xqO /+9xMHdK4OPx1tjDcYzUMu044TcfJ4E= X-MC-Unique: shPCPP4PMDCew2l_6hu4GA-1 From: Michal Privoznik To: libvir-list@redhat.com Subject: [PATCH v6 16/16] kbase: Document virtio-mem Date: Tue, 21 Sep 2021 15:51:32 +0200 Message-Id: <05cdc6fb5f5ad7b8e0439a44202d40e23738c7fb.1632231694.git.mprivozn@redhat.com> In-Reply-To: References: MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 X-loop: libvir-list@redhat.com Cc: david@redhat.com X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=libvir-list-bounces@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1632232446440100001 Content-Type: text/plain; charset="utf-8" This commit adds new memorydevices.rst page which should serve all models of memory devices. Yet, I'm documenting virtio-mem quirks only. Signed-off-by: Michal Privoznik --- docs/kbase/index.rst | 4 + docs/kbase/memorydevices.rst | 150 +++++++++++++++++++++++++++++++++++ docs/kbase/meson.build | 1 + 3 files changed, 155 insertions(+) create mode 100644 docs/kbase/memorydevices.rst diff --git a/docs/kbase/index.rst b/docs/kbase/index.rst index 351ea2c93b..09b19ed1af 100644 --- a/docs/kbase/index.rst +++ b/docs/kbase/index.rst @@ -52,6 +52,10 @@ Usage `PCI topology <../pci-addresses.html>`__ Addressing schemes for PCI devices =20 +`Memory devices `__ + Memory devices and their use + + Internals / Debugging --------------------- =20 diff --git a/docs/kbase/memorydevices.rst b/docs/kbase/memorydevices.rst new file mode 100644 index 0000000000..7f58a4247b --- /dev/null +++ b/docs/kbase/memorydevices.rst @@ -0,0 +1,150 @@ +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D +Memory devices +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D + +.. contents:: + +Basics +=3D=3D=3D=3D=3D=3D + +Memory devices can be divided into two families: volatile and non-volatile. +The former is typical RAM memory: it's volatile and thus its contents does= n't +survive guest reboots or power cycles. The latter retains its contents acr= oss +reboots or power outages. + +In Libvirt, there are two models for volatile memory: + +* ``dimm`` model: + + :: + + + + 523264 + 0 + +
+ + +* ``virtio-mem`` model: + + :: + + + + 1048576 + 0 + 2048 + 524288 + +
+ + +Then there are two models for non-volatile memory: + +* ``nvdimm`` model: + + :: + + + + /tmp/nvdimm + + + 523264 + 0 + +
+ + +* ``virtio-pmem`` model: + + :: + + + + /tmp/virtio_pmem + + + 524288 + +
+ + + +Please note that (maybe somewhat surprisingly) virtio models go onto PCI b= us +instead of DIMM slots. + +Furthermore, DIMMs can have ```` element which configures backend= for +devices. For NVDIMMs the element is mandatory and reflects where the conte= nt +is saved. + +See `memory devices documentation <../formatdomain.html#elementsMemory>`_. + +``virtio-mem`` model +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D + +The ``virtio-mem`` model can be viewed as revised memory balloon. It offers +adding and removing memory (without the actual hotplug of the device). It +solves problems that memory balloon can't solve on its own and thus is more +flexible than DIMM + balloon solution. ``virtio-mem`` is NUMA aware, and t= hus +memory can be inflated/deflated only for a subset of guest NUMA nodes. Al= so, +it works with chunks that are either exposed to guest or reclaimed from it. + +See https://virtio-mem.gitlab.io/ + +Under the hood, ``virtio-mem`` device is split into chunks of equal size w= hich +are then exposed to the guest. Either all of them or only a portion depend= ing +on user's request. Therefore there are three important sizes for +``virtio-mem``. All are to be found under ```` element: + +#. The maximum size the device can ever offer, exposed under ```` +#. The size of a single block, exposed under ```` +#. The current size exposed to the guest, exposed under ```` + +For instance, in the following example the maximum size is 4GiB, the block= size +is 2MiB and only 1GiB should be exposed to the guest: + + :: + + + + 4194304 + 2048 + 1048576 + + + +Please note that ```` must be an integer multiple of ```` +size or zero (no blocks exposed to the guest) and has to be less or equal = to +```` (all blocks exposed to the guest). Furthermore, QEMU recommend= s the +```` size to be as big as a Transparent Huge Page (usually 2MiB). + +To change the size exposed to the guest, users should pass memory device X= ML +with nothing but ```` changed into the +``virDomainUpdateDeviceFlags()`` API. For user's convenience this can be d= one +via virsh too: + + :: + + # virsh update-memory-device $dom --requested-size 2GiB + +If there are two or more ```` devices then ``--alias`` shall be u= sed +to tell virsh which memory device should be updated. + +For running guests there is fourth size that can be found under ````: + + :: + + 2097152 + +The ```` reflects the current size used by the guest. In general= it +can differ from ````. Reasons include guest kernel missing +``virtio-mem`` module and thus being unable to take offered memory, or gue= st +kernel being unable to free memory. Since ```` only reports siz= e to +users, the element is never parsed. It is formatted only into live XML. + +Since changing ```` allocation requires cooperation with guest +kernel, requests for change are not instant. Therefore, libvirt emits +``VIR_DOMAIN_EVENT_ID_MEMORY_DEVICE_SIZE_CHANGE`` event whenever current +allocation changed. diff --git a/docs/kbase/meson.build b/docs/kbase/meson.build index 73dc11837f..af067db453 100644 --- a/docs/kbase/meson.build +++ b/docs/kbase/meson.build @@ -10,6 +10,7 @@ docs_kbase_files =3D [ 'locking-lockd', 'locking', 'locking-sanlock', + 'memorydevices', 'merging_disk_image_chains', 'migrationinternals', 'qemu-core-dump', --=20 2.32.0