From nobody Sun Feb 8 21:37:12 2026 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of redhat.com designates 170.10.133.124 as permitted sender) client-ip=170.10.133.124; envelope-from=libvir-list-bounces@redhat.com; helo=us-smtp-delivery-124.mimecast.com; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1619184340; cv=none; d=zohomail.com; s=zohoarc; b=UzhMNPLtbWz/R/N2DwYxz4tWsev/KI1ZPWTMPiawexbON5Ru8W2Cp6keFVi5zNCVRy7McWu8lzZuUSBQvVt0jNRmr41nBD6p2PdSc6VioszINkrLkBeeTHmGR2u0yv94BjnPj0AuQNCQ7uNOzhVSpjVJ5Ctqnzq7qoFMymgiWwk= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1619184340; h=Content-Type:Content-Transfer-Encoding:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=Jqvn/vkqpqa/fsHYencOJY4Os753zW2l4HHE75TosgY=; b=MPszfBDlCxiRSBfOGYLLRDvJekYTCZp7ZdUaS5hPD0zWf0ir+MPEOQdcWQd0qG+7xdr7DDlz2P+AsQ0BefhX0BaHSiwMBkEXOR8UrrckfUTCUSpbKspKjJjMgK42M2QEBLtJkQ4dFY9BTU+gaJREbuvrKBCgkRKSrK7KUUqKyT8= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass header.from= (p=none dis=none) header.from= Return-Path: Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by mx.zohomail.com with SMTPS id 1619184339962810.8842050750463; Fri, 23 Apr 2021 06:25:39 -0700 (PDT) Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-222-v7wriE4PNgqFGpJ3mKfnRw-1; Fri, 23 Apr 2021 09:25:35 -0400 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 667378030CA; Fri, 23 Apr 2021 13:25:30 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.21]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 4486D1B5AB; Fri, 23 Apr 2021 13:25:30 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id 0120044A6B; Fri, 23 Apr 2021 13:25:29 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id 13NDPSCH022007 for ; Fri, 23 Apr 2021 09:25:28 -0400 Received: by smtp.corp.redhat.com (Postfix) id 2058763634; Fri, 23 Apr 2021 13:25:28 +0000 (UTC) Received: from localhost.localdomain (unknown [10.40.195.9]) by smtp.corp.redhat.com (Postfix) with ESMTP id 9183B772E1 for ; Fri, 23 Apr 2021 13:25:24 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1619184338; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post; bh=Jqvn/vkqpqa/fsHYencOJY4Os753zW2l4HHE75TosgY=; b=ilO4y55QembQWc1Oqtg3UveH9QxQgmqx0W82h4QPLli03cAz0dgmdN8wl/FKQL8qFPX68d MV4+1flIo6HIdkOp60Y+mQyFSPjhqowcoVIb/gtpTyH7LLYjvfOUCTY6JiIw+DATebbrJq swl+EQVQ9ubIsrTX5Yw9PN7nT7iBZ+E= X-MC-Unique: v7wriE4PNgqFGpJ3mKfnRw-1 From: Michal Privoznik To: libvir-list@redhat.com Subject: [PATCH v3 14/14] kbase: Document virtio-mem Date: Fri, 23 Apr 2021 15:24:36 +0200 Message-Id: In-Reply-To: References: MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-loop: libvir-list@redhat.com X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=libvir-list-bounces@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @redhat.com) Content-Type: text/plain; charset="utf-8" This commit adds new memorydevices.rst page which should serve all models of memory devices. Yet, I'm documenting virtio-mem quirks only. Signed-off-by: Michal Privoznik --- docs/kbase/index.rst | 4 + docs/kbase/memorydevices.rst | 150 +++++++++++++++++++++++++++++++++++ docs/kbase/meson.build | 1 + 3 files changed, 155 insertions(+) create mode 100644 docs/kbase/memorydevices.rst diff --git a/docs/kbase/index.rst b/docs/kbase/index.rst index 532804fe05..45450bf33b 100644 --- a/docs/kbase/index.rst +++ b/docs/kbase/index.rst @@ -46,6 +46,10 @@ Usage `PCI topology <../pci-addresses.html>`__ Addressing schemes for PCI devices =20 +`Memory devices `__ + Memory devices and their use + + Internals / Debugging --------------------- =20 diff --git a/docs/kbase/memorydevices.rst b/docs/kbase/memorydevices.rst new file mode 100644 index 0000000000..5c4c45a77f --- /dev/null +++ b/docs/kbase/memorydevices.rst @@ -0,0 +1,150 @@ +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D +Memory devices +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D + +.. contents:: + +Basics +=3D=3D=3D=3D=3D=3D + +Memory devices can be divided into two families: DIMMs and NVDIMMs. The fo= rmer +is typical RAM memory: it's volatile and thus its contents doesn't survive +reboots nor guest shut downs and power ons. The latter retains its contents +across reboots or power outages. + +In Libvirt, there are two models for DIMMs: + +* ``dimm`` model: + + :: + + + + 523264 + 0 + +
+ + +* ``virtio-mem`` model: + + :: + + + + 1048576 + 0 + 2048 + 524288 + +
+ + +Then there are two models for NVDIMMs: + +* ``nvidmm`` model: + + :: + + + + /tmp/nvdimm + + + 523264 + 0 + +
+ + +* ``virtio-pmem`` model: + + :: + + + + /tmp/virtio_pmem + + + 524288 + +
+ + + +Please note that (maybe somewhat surprisingly) virtio models go onto PCI b= us +instead of DIMM slots. + +Furthermore, DIMMs can have ```` element which configures backend= for +devices. For NVDIMMs the element is mandatory and reflects where the conte= nts +is saved. + +See `memory devices documentation <../formatdomain.html#elementsMemory>`_. + +``virtio-mem`` model +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D + +The ``virtio-mem`` model can be viewed as revised memory balloon. It offers +adding and removing memory (without the actual hotplug of the device). It +solves problems that memory balloon can't solve on its own and thus is more +flexible than DIMM + balloon solution. ``virtio-mem`` is NUMA aware, and t= hus +memory can be inflated/deflated only for a subset of guest NUMA nodes. Al= so, +it works with chunks that are either exposed to guest or taken back from i= t. + +See https://virtio-mem.gitlab.io/ + +Under the hood, ``virtio-mem`` device is split into chunks of equal size w= hich +are then exposed to the guest. Either all of them or only a portion depend= ing +on user's request. Therefore there are three important sizes for +``virtio-mem``. All are to be found under ```` element: + +#. The maximum size the device can ever offer, exposed under ```` +#. The size of a single block, exposed under ```` +#. The current size exposed to the guest, exposed under ```` + +For instance, the following example the maximum size is 4GiB, the block si= ze is +2MiB and only 1GiB should be exposed to the guest: + + :: + + + + 4194304 + 2048 + 1048576 + + + +Please note that ```` must be an integer multiple of ```` +size or zero (no blocks exposed to the guest) and has to be less or equal = to +```` (all blocks exposed to the guest). Furthermore, QEMU recommend= s the +```` size to be as big as a Transparent Huge Page (usually 2MiB). + +To change the size exposed to the guest, users should pass memory device X= ML +with nothing but ```` changed into the +``virDomainUpdateDeviceFlags()`` API. For user's convenience this can be d= one +via virsh too: + + :: + + # virsh update-memory-device $dom --requested size 2GiB + +If there are two or more ```` devices then ``--alias`` shall be u= sed +to tell virsh which memory device should be updated. + +For running guests there is fourth size that can be found under ````: + + :: + + 2097152 + +The ```` reflects the actual size used by the guest. In general it +can differ from ````. Reasons include guest kernel missing +``virtio-mem`` module and thus being unable to take offered memory, or gue= st +kernel being unable to free memory. Since ```` only reports size= to +users, the element is never parsed. It is formatted only into live XML. + +Since changing actual allocation requires cooperation with guest kernel, +requests for change are not instant. Therefore, libvirt emits +``VIR_DOMAIN_EVENT_ID_MEMORY_DEVICE_SIZE_CHANGE`` event whenever actual +allocation changed. diff --git a/docs/kbase/meson.build b/docs/kbase/meson.build index 7b4e7abbd3..7f6adba915 100644 --- a/docs/kbase/meson.build +++ b/docs/kbase/meson.build @@ -9,6 +9,7 @@ docs_kbase_files =3D [ 'locking-lockd', 'locking', 'locking-sanlock', + 'memorydevices', 'migrationinternals', 'qemu-passthrough-security', 'rpm-deployment', --=20 2.26.3