From nobody Wed May 1 07:16:26 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of redhat.com designates 209.132.183.28 as permitted sender) client-ip=209.132.183.28; envelope-from=libvir-list-bounces@redhat.com; helo=mx1.redhat.com; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1569339222; cv=none; d=zoho.com; s=zohoarc; b=XqsMqpWP9Ro1Y4bdArPZdM0yM5zFSQJACBR6tZKZk7YHBsLI/cF8F2GjFqgQB9G5rk6Dhd/LbeZiCsnMNu5QxTjlN59oiQTnDEOCin4VUQ9uVEoZc16pVO1J1yloxG52bvRmUqg3O4qVFDRiivU1kwUkTGUdvAzu7/3dzaP8Ln0= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1569339222; h=Content-Type:Content-Transfer-Encoding:Date:From:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:Sender:Subject:To:ARC-Authentication-Results; bh=t4WwV4fBNpKBj5FT42yec37B2TKm1OTHPHY0QBaeBtk=; b=ZS2c8VjuV+GpsxCddg/V3XrMDCfPbaDB87CCC39SN21zkAuTuemtWadZCaFlzzBIC4F7hWg8cvytiTz6KkjNubNBtbWSuoPaFsOdjGVg4DQlc6voJndJlJ6/GK/loqTYAe/qVWhwevBDAqTw2Qwd+QwJnskztg4kFsVGp7a3Nds= ARC-Authentication-Results: i=1; mx.zoho.com; spf=pass (zoho.com: domain of redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass header.from= (p=none dis=none) header.from= Return-Path: Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by mx.zohomail.com with SMTPS id 15693392222201005.2122885057048; Tue, 24 Sep 2019 08:33:42 -0700 (PDT) Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.phx2.redhat.com [10.5.11.22]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 6FEE6309BF21; Tue, 24 Sep 2019 15:33:40 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.21]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 4824D10013D9; Tue, 24 Sep 2019 15:33:40 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id D3B7D6B49E; Tue, 24 Sep 2019 15:33:39 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id x8OFXc4U030409 for ; Tue, 24 Sep 2019 11:33:38 -0400 Received: by smtp.corp.redhat.com (Postfix) id CD325600CC; Tue, 24 Sep 2019 15:33:38 +0000 (UTC) Received: from localhost.localdomain.com (ovpn-112-39.ams2.redhat.com [10.36.112.39]) by smtp.corp.redhat.com (Postfix) with ESMTP id EF70F600C8; Tue, 24 Sep 2019 15:33:35 +0000 (UTC) From: =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= To: libvir-list@redhat.com Date: Tue, 24 Sep 2019 16:33:33 +0100 Message-Id: <20190924153333.977-1-berrange@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 X-loop: libvir-list@redhat.com Subject: [libvirt] [PATCH v2] docs: attempt to document the general libvirt dev strategy X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-Scanned-By: MIMEDefang 2.84 on 10.5.11.22 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.45]); Tue, 24 Sep 2019 15:33:41 +0000 (UTC) There are various ideas / plans floating around for future libvirt work, some of which is actively in progress. Historically we've never captured this kind of information anywhere, except in mailing list discussions. In particular guidelines in hacking.html.in don't appear until a policy is actively applied. This patch attempts to fill the documentation gap, by creating a new "strategy" page which outlines the general vision for some notable future changes. The key thing to note is that none of the stuff on this page is guaranteed, plans may change as new information arises. IOW this is a "best guess" as to the desired future. This doc has focused on three areas, related to the topic of language usage / consolidation - Use of non-C languages for the library, daemons or helper tools - Replacement of autotools with meson - Use of RST and Sphinx for documentation (website + man pages) Signed-off-by: Daniel P. Berrang=C3=A9 Reviewed-by: Michal Privoznik --- docs/docs.html.in | 3 + docs/strategy.html.in | 144 ++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 147 insertions(+) create mode 100644 docs/strategy.html.in diff --git a/docs/docs.html.in b/docs/docs.html.in index c1741c89b4..6cf09f51bc 100644 --- a/docs/docs.html.in +++ b/docs/docs.html.in @@ -128,6 +128,9 @@
Contributor guidelines
General hacking guidelines for contributors
=20 +
Project strategy
+
Sets a vision for future direction & technical choices
+
Bug reports
How and where to report bugs and request features
=20 diff --git a/docs/strategy.html.in b/docs/strategy.html.in new file mode 100644 index 0000000000..2ed5deeec6 --- /dev/null +++ b/docs/strategy.html.in @@ -0,0 +1,144 @@ + + + + +

Libvirt Project Strategy

+ +

+ This document attempts to outline the libvirt project strategy for + the near future. Think of this as a high level vision or todo list + setting the direction for the project and its developers to take. +

+ +

Language consolidation

+ +

+ At time of writing libvirt uses the following languages +

+ +
+
C
+
The core libvirt library, daemons, and helper tools are all writ= ten + in the C language.
+
Python
+
Various supporting build/test scripts are written in Python, with + compatibility for Python 2 and 3.
+
Perl
+
Various supporting build/test scripts are written in Perl. It is + also used for many syntax-check inline rules
+
Shell
+
The autoconf configure script, is essentially shell script. Shell + is also used for some simple build/test scripts. At runtime libvirt + avoids shell except when using SSH tunnels to a remote host
+
XSLT
+
The website uses XSLT for its templating system. The API + documentation is also autogenerated from an XML description + using XSLT
+
HTML
+
The website documentation is all written in plain HTML. Some HTML + is also auto-generated for API documentation
+
M4
+
The autoconf configure script uses a large number of M4 macros to + generate its content
+
make
+
The core build system uses the traditional GNU make recipes
+
automake
+
The make recipes use automake's language extensions which are + then turned into regular make rules
+
awk/sed
+
A number of the syntax-check inline rules involve use of awk/sed + scripts
+
POD
+
The command line manual pages are typically written in Perl's POD + format, and converted to troff
+
+ +

+ The wide range of languages used present a knowledge burden for + developers involved in libvirt, especially when there are multiple + languages all used in the same problem spaces. This is most notable + in the build system which uses a combination of shell, m4, make, + automake, awk, sed, perl and python, with debugging requiring + understanding of the interactions between many languages. The + relative popularity of the languages has implications for how easily + the project can attract new contributors, and the quality of the code + they'll be able to submit. This is most notable when comparing Perl + to Python, as since the start of the libvirt project, the popularity + and knowledge of Perl has declined, while Python has risen. +

+

+ The C language has served libvirt well over the years, but its age s= hows + giving rise to limitations which negatively impact the project in te= rms + of code quality, reliability, and efficiency of development. Most no= tably + its lack of memory safety means that many code bugs become trivially + exploitable security flaws or denial of service. The lack of a high + level portable runtime, resulting in a lot of effort being spent to + ensure cross platform portability. The modern languages Rust and Go + provide viable options for low level systems programming, in a way t= hat + was not practical with other common languages such as Python and Jav= a. + There is thus a desire to make use of either Rust or Go, or a combin= ation + of both, to incrementally replace existing use of C, and also for + greenfield development. +

+

+ With this in mind the libvirt project has set a vision for language + usage in the future +

+ +
+
C
+
Large parts of the core libvirt library, daemons, and helper too= ls + will continue to make use in the C language. Integration of other + languages will be an incremental, targetted process where they can + bring the greatest benefit.
+
Rust / Go
+
Parts of the core libvirt library, daemons and helper tools are = to + leverage Rust or Go or both to replace C.
+
Meson
+
The core build system is to be written in meson.
+
Python
+
Various supporting build/test scripts are written in Python 3 + compatible mode only.
+
reStructuredText
+
The website and command man pages are to be written in RST, using + Sphinx as the engine to convert to end user formats like HTML, tro= ff, + etc
+
+ +

+ Some notable points from the above. Whether the core library / daemo= ns + will use Rust or Go internally is still to be decided based on more + detailed evaluation to identify the best fit. The need to link and e= mbed + this functionality in other processes has complex interactions both = at a + technical and non-technical level. For standalone helper tools, eith= er + language is viable, but there are fewer concerns around interactions= with + other in-process code from 3rd parties. Thus a different decision ma= y be + made for daemons/libraries vs tools. Any rewrite proposed for existi= ng + functionality will have to weigh up the benefits of the new code, + against the risk of introducing regressions wrt the previous code. +

+ +

+ The meson build system is written in Python 3. This directly informs= the + choice of Python 3 as the language for all supporting build scripts, + re-inforcing the other benefits of Python, over Perl, Shell, M4, + automake, etc. There is no intention to support Python 2 given meson= 's + requirement for Python 3. +

+ +

+ Using the RST format for documentation allows for the use of XSLT to= be + eliminated from the build process. RST and the Sphinx toolkit are wi= dely + used, as seen by the huge repository of content on + https://readthedocs.org/. The ability to embed raw HTML in the RST d= ocs + will greatly facilitate its adoption, avoiding the need for a big ba= ng + conversion of existing content. Given the desire to eliminate Perl + usage, replacing the use of POD documentation for manual pages is an + obvious followup task. RST is the obvious choice to achieve alignment + with the website, allowing the man pages to be easily published onli= ne + with other docs. It is further anticipated that the current API docs + generator which uses XSLT to convert the XML API description would be + converted to something which generates RST using Python instead of X= SLT. +

+ + --=20 2.21.0 -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list