From nobody Tue Feb 10 11:14:46 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=none (zoho.com: 192.237.175.120 is neither permitted nor denied by domain of lists.xenproject.org) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org ARC-Seal: i=1; a=rsa-sha256; t=1570136267; cv=none; d=zoho.com; s=zohoarc; b=i2RgCHrvGlW2DGdlqSsODWGl3EIr5YkSUfSvYvkxZHiCkMlWxNYNa/ThjGI5+OZTVZSUlnvOMMl5wERdXwz9vDZiPqAvkeiV+rYtWXuW1aP3abPnYVUIxMKoIXg4yLhj2XRSZGs9B9c7PtFqOSqE1SGloKJIuQEVEFLgsR0xNPs= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1570136267; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To:ARC-Authentication-Results; bh=uiWTEqDW8hAjvocIGhck2pJtWkpmPbSjYK07JjlAIO8=; b=b/J7tJAwpkC9VHG890NgNAWm6aOCrkiCvIbge07+XGctjP3Pl/PpHAwiMnQc47I1qUoJf4e6Hs080ZNNOrV4MLOKndSdtTO+aaWGCNQ/yC6A9Nohyd5/GhzJIHleZsv9M8P4xZIYgabjeJ/PgqaZkZjiRT5mxlqOY76kEomfqvU= ARC-Authentication-Results: i=1; mx.zoho.com; dkim=fail; spf=none (zoho.com: 192.237.175.120 is neither permitted nor denied by domain of lists.xenproject.org) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1570136267865673.5246728384938; Thu, 3 Oct 2019 13:57:47 -0700 (PDT) Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1iG89W-000521-TA; Thu, 03 Oct 2019 20:56:46 +0000 Received: from us1-rack-iad1.inumbo.com ([172.99.69.81]) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1iG89V-00051d-OX for xen-devel@lists.xenproject.org; Thu, 03 Oct 2019 20:56:45 +0000 Received: from esa3.hc3370-68.iphmx.com (unknown [216.71.145.155]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS id 493e68aa-e620-11e9-80e3-bc764e2007e4; Thu, 03 Oct 2019 20:56:35 +0000 (UTC) X-Inumbo-ID: 493e68aa-e620-11e9-80e3-bc764e2007e4 DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=citrix.com; s=securemail; t=1570136196; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=GC3HAQyto0ckX2WRYwQnR+QUL/z42b/o2YvqwmH7ryI=; b=RN7xuyK2rFd38uXER9L4YTtt1htBushq50uy6vBBo0gU29MCIsYcDpln MjDB7WbKf6yofItEoSebiugM6Cba7hBmD2eFgJtnayT7IQfqX05NnGGjU uuYGp2QIwbSAi77XElpXHD6VylqohikGWcWiY0dhEjVqvvX6+taKw5fz2 c=; Authentication-Results: esa3.hc3370-68.iphmx.com; dkim=none (message not signed) header.i=none; spf=None smtp.pra=andrew.cooper3@citrix.com; spf=Pass smtp.mailfrom=Andrew.Cooper3@citrix.com; spf=None smtp.helo=postmaster@mail.citrix.com Received-SPF: none (zoho.com: 192.237.175.120 is neither permitted nor denied by domain of lists.xenproject.org) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Received-SPF: None (esa3.hc3370-68.iphmx.com: no sender authenticity information available from domain of andrew.cooper3@citrix.com) identity=pra; client-ip=162.221.158.21; receiver=esa3.hc3370-68.iphmx.com; envelope-from="Andrew.Cooper3@citrix.com"; x-sender="andrew.cooper3@citrix.com"; x-conformance=sidf_compatible Received-SPF: Pass (esa3.hc3370-68.iphmx.com: domain of Andrew.Cooper3@citrix.com designates 162.221.158.21 as permitted sender) identity=mailfrom; client-ip=162.221.158.21; receiver=esa3.hc3370-68.iphmx.com; envelope-from="Andrew.Cooper3@citrix.com"; x-sender="Andrew.Cooper3@citrix.com"; x-conformance=sidf_compatible; x-record-type="v=spf1"; x-record-text="v=spf1 ip4:209.167.231.154 ip4:178.63.86.133 ip4:195.66.111.40/30 ip4:85.115.9.32/28 ip4:199.102.83.4 ip4:192.28.146.160 ip4:192.28.146.107 ip4:216.52.6.88 ip4:216.52.6.188 ip4:162.221.158.21 ip4:162.221.156.83 ~all" Received-SPF: None (esa3.hc3370-68.iphmx.com: no sender authenticity information available from domain of postmaster@mail.citrix.com) identity=helo; client-ip=162.221.158.21; receiver=esa3.hc3370-68.iphmx.com; envelope-from="Andrew.Cooper3@citrix.com"; x-sender="postmaster@mail.citrix.com"; x-conformance=sidf_compatible IronPort-SDR: +Bq5ngUqU4qhDJAGm+8IptJuM1TSGOivXbAUoKPgHhMEa0u0CSJDD/WHasw5MX5QvrKM6NSdIi vZ/OyLoJ5ICwecomRsz/1oK2GomxmgJPOl/W9PU8y2tedNeCboS360TthldbuKSpRTgbDbSU4a sVCEKlsgLAgBtIG90bMgyMc1lt8Nwyq39bRUkdGMBywuOireTgvEAQVRUZ2J5xh8O3KYOIP7Wv GrpOYI9TzJ8p0ecjaMx3u0MWTJO2tyLMmCIG0rNcq+hqsjys45WyBNfktOuhDpWDMUhjhTU5OA idY= X-SBRS: 2.7 X-MesageID: 6447655 X-Ironport-Server: esa3.hc3370-68.iphmx.com X-Remote-IP: 162.221.158.21 X-Policy: $RELAYED X-IronPort-AV: E=Sophos;i="5.67,253,1566878400"; d="scan'208";a="6447655" From: Andrew Cooper To: Xen-devel Date: Thu, 3 Oct 2019 21:56:23 +0100 Message-ID: <20191003205623.20839-4-andrew.cooper3@citrix.com> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20191003205623.20839-1-andrew.cooper3@citrix.com> References: <20191003205623.20839-1-andrew.cooper3@citrix.com> MIME-Version: 1.0 Subject: [Xen-devel] [PATCH 4/4] docs/sphinx: Technical Debt X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Lars Kurth , Stefano Stabellini , Julien Grall , Wei Liu , Konrad Rzeszutek Wilk , George Dunlap , Andrew Cooper , Tim Deegan , Jan Beulich , Ian Jackson , =?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) This identifies various of areas technical debt, which either need to be, or are being worked on, along with enough clarifying details for people to follow. Signed-off-by: Andrew Cooper --- CC: Lars Kurth CC: George Dunlap CC: Ian Jackson CC: Jan Beulich CC: Konrad Rzeszutek Wilk CC: Stefano Stabellini CC: Tim Deegan CC: Wei Liu CC: Julien Grall CC: Roger Pau Monn=C3=A9 --- docs/conf.py | 11 +++- docs/index.rst | 8 +++ docs/misc/tech-debt.rst | 130 ++++++++++++++++++++++++++++++++++++++++++++= ++++ 3 files changed, 148 insertions(+), 1 deletion(-) create mode 100644 docs/misc/tech-debt.rst diff --git a/docs/conf.py b/docs/conf.py index 50e41501db..0d2227f52e 100644 --- a/docs/conf.py +++ b/docs/conf.py @@ -53,7 +53,7 @@ # Add any Sphinx extension module names here, as strings. They can be # extensions coming with Sphinx (named 'sphinx.ext.*') or your custom # ones. -extensions =3D [] +extensions =3D ['sphinx.ext.extlinks'] =20 # Add any paths that contain templates here, relative to this directory. templates_path =3D ['_templates'] @@ -192,3 +192,12 @@ =20 # A list of files that should not be packed into the epub file. epub_exclude_files =3D ['search.html'] + + +# -- Configuration for external links ------------------------------------= ---- + +extlinks =3D { + 'xen-cs': + ('https://xenbits.xen.org/gitweb/?p=3Dxen.git;a=3Dcommitdiff;h=3D%= s', + 'Xen c/s '), +} diff --git a/docs/index.rst b/docs/index.rst index b8ab13178c..0a2af2db9d 100644 --- a/docs/index.rst +++ b/docs/index.rst @@ -59,3 +59,11 @@ Miscellanea .. toctree:: =20 glossary + +Unsorted +-------- + +.. toctree:: + :maxdepth: 2 + + misc/tech-debt diff --git a/docs/misc/tech-debt.rst b/docs/misc/tech-debt.rst new file mode 100644 index 0000000000..172ba3bd51 --- /dev/null +++ b/docs/misc/tech-debt.rst @@ -0,0 +1,130 @@ +.. SPDX-License-Identifier: CC-BY-4.0 + +Technical Debt +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D + +Hypervisor +---------- + +CONFIG_PDX +~~~~~~~~~~ + +Xen uses the term MFN for Machine Frame Number, which is synonymous with +Linux's PFN, and maps linearly to system/host/machine physical addresses. + +For every page of RAM, a ``struct page_info`` is needed for tracking purpo= ses. +In the simple case, the frametable is an array of ``struct page_info[]`` +indexed by MFN. + +However, this is inefficient when a system has banks of RAM at spread out = in +address space, as a large amount of space is wasted on frametable entries = for +non-existent frames. This wastes both virtual address space and RAM. + +As a consequence, Xen has a compression scheme known as PDX which removes +unused bits out of the middle of MFNs, to make a more tightly packed Page +inDeX, which in turn reduces the size of the frametable for system. + +At the moment, PDX compression is unconditionally used. + +However, PDX compression does come with a cost in terms of the complexity = to +convert between PFNs and pages, which is a common operation in Xen. + +Typically, ARM32 systems do have RAM banks in discrete locations, and want= to +use PDX compression, while typically ARM64 and x86 systems have RAM packed +from 0 with no holes. + +The goal of this work is to have ``CONFIG_PDX`` selected by ARM32 only. T= his +requires slightly untangling the memory management code in ARM and x86 to = give +it a clean compile boundary where PDX conversions are used. + + +Waitqueue infrastructure +~~~~~~~~~~~~~~~~~~~~~~~~ + +Livepatching safety in Xen depends on all CPUs rendezvousing on the return= to +guest path, with no stack frame. The vCPU waitqueue infrastructure underm= ines +this safety by copying a stack frame sideways, and ``longjmp()``\-ing away. + +Waitqueues are only used by the introspection/mem_event/paging infrastruct= ure, +where the design of the rings causes some problems. There is a single 4k = page +used for the ring, which serves both synchronous requests, and lossless as= ync +requests. In practice, introspecting an 11-vcpu guest is sufficient to ca= use +the waitqueue infrastructure to start to be used. + +A better design of ring would be to have a slot per vcpu for synchronous +requests (simplifies producing and consuming of requests), and a multipage +ring buffer (of negotiable size) with lossy semantics for async requests. + +A design such as this would guarantee that Xen never has to block waiting = for +userspace to create enough space on the ring for a vcpu to write state out. + +.. note:: + + There are other aspects of the existing ring infrastructure which are + driving a redesign, but these don't relate directly to the waitqueue + infrastructure and livepatching safety. + + The most serious problem is that the ring infrastructure is GFN based, + which leaves the guest either able to mess with the ring, or a shattered + host superpage where the ring used to be, and the guest balloon driver = able + to prevent the introspection agent from connecting/reconnecting the rin= g. + +As there are multiple compelling reasons to redesign the ring infrastructu= re, +the plan is to introduce the new ring ABI, deprecate and remove the old AB= I, +and simply delete the waitqueue infrastructure at that point, rather than = try +to redesign livepatching from scratch in an attempt to cope with unwinding= old +stack frames. + + +Dom0 +---- + +Remove xenstored's dependencies on unstable interfaces +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Various xenstored implementations use libxc for two purposes. It would be= a +substantial advantage to move xenstored onto entirely stable interfaces, w= hich +disconnects it from the internal of the libxc. + +1. Foreign mapping of the store ring + + This is obsolete since :xen-cs:`6a2de353a9` (2012) which allocated grant + entries instead, to allow xenstored to function as a stub-domain withou= t dom0 + permissions. :xen-cs:`38eeb3864d` dropped foreign mapping for cxenstor= ed. + However, there are no OCaml bindings for libxengnttab. + + Work Items: + + * Minimal ``tools/ocaml/libs/xg/`` binding for ``tools/libs/gnttab/``. + * Replicate :xen-cs:`38eeb3864d` for oxenstored as well. + +2. Figuring out which domain(s) have gone away + + Currently, the handling of domains is asymmetric. + + * When a domain is created, the toolstack explicitly sends an + ``XS_INTRODUCE(domid, store mfn, store evtchn)`` message to xenstored= , to + cause xenstored to connect to the guest ring, and fire the + ``@introduceDomain`` watch. + + * When a domain is destroyed, Xen fires ``VIRQ_DOM_EXC`` which is bound= by + xenstored, rather than the toolstack. xenstored updates its idea of = the + status of domains, and fires the ``@releaseDomain`` watch. + + Xenstored uses ``xc_domain_getinfo()``, to work out which domain(s) h= ave gone + away, and only cares about the shutdown status. + + Furthermore, ``@releaseDomain`` (like ``VIRQ_DOM_EXC``) is a single-b= it + message, which requires all listeners to evaluate whether the message= applies + to them or not. This results in a flurry of ``xc_domain_getinfo()`` = calls + from multiple entities in the system, which all serialise on the domc= tl lock + in Xen. + + Work Items: + + * Figure out how shutdown status can be expressed in a stable way fro= m Xen. + * Figure out if ``VIRQ_DOM_EXC`` and ``@releaseDomain`` can be extend= ed + or superseded to carry at least a domid, to make domain shutdown sc= ale + better. + * Figure out if ``VIRQ_DOM_EXC`` would better be bound by the toolsta= ck, + rather than xenstored. --=20 2.11.0 _______________________________________________ Xen-devel mailing list Xen-devel@lists.xenproject.org https://lists.xenproject.org/mailman/listinfo/xen-devel