From nobody Sun Feb 8 23:41:39 2026 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of redhat.com designates 170.10.133.124 as permitted sender) client-ip=170.10.133.124; envelope-from=libvir-list-bounces@redhat.com; helo=us-smtp-delivery-124.mimecast.com; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1624443191; cv=none; d=zohomail.com; s=zohoarc; b=TrVaDGGoWwoq7U3q3daH7RoGArawdE6N4hBzZHE2I//LT4drME+DjGzZN0v92H9XyqSo81gGBNoNRW7UmhyQ/RE796CMpMmqs+uPBa5MSN7h8/ebbpuzKuf81TmXT5fLkXMRqB7Ru1/9yqg14WHAif1Xdn9G6rtEUcZrzfo2qQ8= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1624443191; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=erIpqguxtwsz55JCyG+8rMQKKb5/idyvquvfCmEA3kw=; b=geNYHJCj9UmNqwnFnXaOPEOFZyT35glppCrovdGXCluuQzwGoGwV3euNF1tmJLzpQ6/jxCYxwiccmX1mHVkETLYNP4iHRCKGSw3zLlE9QbXQMBgiqOJyEckOXfhGpdegHRrAlA9v37Tl968VCPvO5nbbGexUe8fnhlTtknjXIKo= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by mx.zohomail.com with SMTPS id 1624443191920765.74371296001; Wed, 23 Jun 2021 03:13:11 -0700 (PDT) Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-466-Y9kVhP2JNvSW_ZC2js4Ogg-1; Wed, 23 Jun 2021 06:13:09 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id A7FBC101C8B1; Wed, 23 Jun 2021 10:13:02 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.20]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 811625D6D7; Wed, 23 Jun 2021 10:13:02 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id 45E9C18199FF; Wed, 23 Jun 2021 10:13:02 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.phx2.redhat.com [10.5.11.23]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id 15NACx5t021121 for ; Wed, 23 Jun 2021 06:12:59 -0400 Received: by smtp.corp.redhat.com (Postfix) id 099C019D9D; Wed, 23 Jun 2021 10:12:59 +0000 (UTC) Received: from localhost.localdomain (unknown [10.40.194.171]) by smtp.corp.redhat.com (Postfix) with ESMTP id 551C119C66; Wed, 23 Jun 2021 10:12:58 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1624443190; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post; bh=erIpqguxtwsz55JCyG+8rMQKKb5/idyvquvfCmEA3kw=; b=IZLETSL6ssujAw6dtGhi7EJwjrD2lb0mzbVeHXXogwoVlZ5hFc64cL1RK5Dz62k3ppMCxs j7ussxdc1FCOi2BjFN2JRTg50ymKrdObWTTqOq8J9+oHd8X7UXNF2RjQMiAfJH8cDCC09N JC+m5Bmi4L+INM7bN5Pq6eTujPNpf/o= X-MC-Unique: Y9kVhP2JNvSW_ZC2js4Ogg-1 From: Michal Privoznik To: libvir-list@redhat.com Subject: [PATCH v4 14/14] kbase: Document virtio-mem Date: Wed, 23 Jun 2021 12:12:19 +0200 Message-Id: <1dcca8e67c175a00bf809003580959eeade9dfac.1624442835.git.mprivozn@redhat.com> In-Reply-To: References: MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.23 X-loop: libvir-list@redhat.com Cc: david@redhat.com X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=libvir-list-bounces@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @redhat.com) Content-Type: text/plain; charset="utf-8" This commit adds new memorydevices.rst page which should serve all models of memory devices. Yet, I'm documenting virtio-mem quirks only. Signed-off-by: Michal Privoznik Reviewed-by: Jim Fehlig --- docs/kbase/index.rst | 4 + docs/kbase/memorydevices.rst | 150 +++++++++++++++++++++++++++++++++++ docs/kbase/meson.build | 1 + 3 files changed, 155 insertions(+) create mode 100644 docs/kbase/memorydevices.rst diff --git a/docs/kbase/index.rst b/docs/kbase/index.rst index 91083ee49d..6355fe4f1d 100644 --- a/docs/kbase/index.rst +++ b/docs/kbase/index.rst @@ -52,6 +52,10 @@ Usage `PCI topology <../pci-addresses.html>`__ Addressing schemes for PCI devices =20 +`Memory devices `__ + Memory devices and their use + + Internals / Debugging --------------------- =20 diff --git a/docs/kbase/memorydevices.rst b/docs/kbase/memorydevices.rst new file mode 100644 index 0000000000..23ccd6da88 --- /dev/null +++ b/docs/kbase/memorydevices.rst @@ -0,0 +1,150 @@ +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D +Memory devices +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D + +.. contents:: + +Basics +=3D=3D=3D=3D=3D=3D + +Memory devices can be divided into two families: volatile and non-volatile. +The former is typical RAM memory: it's volatile and thus its contents does= n't +survive reboots nor guest shut downs and power ons. The latter retains its +contents across reboots or power outages. + +In Libvirt, there are two models for volatile memory: + +* ``dimm`` model: + + :: + + + + 523264 + 0 + +
+ + +* ``virtio-mem`` model: + + :: + + + + 1048576 + 0 + 2048 + 524288 + +
+ + +Then there are two models for non-volatile memory: + +* ``nvidmm`` model: + + :: + + + + /tmp/nvdimm + + + 523264 + 0 + +
+ + +* ``virtio-pmem`` model: + + :: + + + + /tmp/virtio_pmem + + + 524288 + +
+ + + +Please note that (maybe somewhat surprisingly) virtio models go onto PCI b= us +instead of DIMM slots. + +Furthermore, DIMMs can have ```` element which configures backend= for +devices. For NVDIMMs the element is mandatory and reflects where the conte= nts +is saved. + +See `memory devices documentation <../formatdomain.html#elementsMemory>`_. + +``virtio-mem`` model +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D + +The ``virtio-mem`` model can be viewed as revised memory balloon. It offers +adding and removing memory (without the actual hotplug of the device). It +solves problems that memory balloon can't solve on its own and thus is more +flexible than DIMM + balloon solution. ``virtio-mem`` is NUMA aware, and t= hus +memory can be inflated/deflated only for a subset of guest NUMA nodes. Al= so, +it works with chunks that are either exposed to guest or taken back from i= t. + +See https://virtio-mem.gitlab.io/ + +Under the hood, ``virtio-mem`` device is split into chunks of equal size w= hich +are then exposed to the guest. Either all of them or only a portion depend= ing +on user's request. Therefore there are three important sizes for +``virtio-mem``. All are to be found under ```` element: + +#. The maximum size the device can ever offer, exposed under ```` +#. The size of a single block, exposed under ```` +#. The current size exposed to the guest, exposed under ```` + +For instance, the following example the maximum size is 4GiB, the block si= ze is +2MiB and only 1GiB should be exposed to the guest: + + :: + + + + 4194304 + 2048 + 1048576 + + + +Please note that ```` must be an integer multiple of ```` +size or zero (no blocks exposed to the guest) and has to be less or equal = to +```` (all blocks exposed to the guest). Furthermore, QEMU recommend= s the +```` size to be as big as a Transparent Huge Page (usually 2MiB). + +To change the size exposed to the guest, users should pass memory device X= ML +with nothing but ```` changed into the +``virDomainUpdateDeviceFlags()`` API. For user's convenience this can be d= one +via virsh too: + + :: + + # virsh update-memory-device $dom --requested-size 2GiB + +If there are two or more ```` devices then ``--alias`` shall be u= sed +to tell virsh which memory device should be updated. + +For running guests there is fourth size that can be found under ````: + + :: + + 2097152 + +The ```` reflects the actual size used by the guest. In general it +can differ from ````. Reasons include guest kernel missing +``virtio-mem`` module and thus being unable to take offered memory, or gue= st +kernel being unable to free memory. Since ```` only reports size= to +users, the element is never parsed. It is formatted only into live XML. + +Since changing actual allocation requires cooperation with guest kernel, +requests for change are not instant. Therefore, libvirt emits +``VIR_DOMAIN_EVENT_ID_MEMORY_DEVICE_SIZE_CHANGE`` event whenever actual +allocation changed. diff --git a/docs/kbase/meson.build b/docs/kbase/meson.build index 7631b47018..f93f687efb 100644 --- a/docs/kbase/meson.build +++ b/docs/kbase/meson.build @@ -10,6 +10,7 @@ docs_kbase_files =3D [ 'locking-lockd', 'locking', 'locking-sanlock', + 'memorydevices', 'merging_disk_image_chains', 'migrationinternals', 'qemu-passthrough-security', --=20 2.31.1