From nobody Tue May 14 01:29:24 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=suse.de ARC-Seal: i=1; a=rsa-sha256; t=1683146451; cv=none; d=zohomail.com; s=zohoarc; b=IqaJ11/HyUTpKB5td52WROYvWlg3fkk4Brcemy9tbA5C47ViwbfdQFA/Ec7u4RXVUpUSipni9a1rdhUrRayzuafzmt9YvD4vmjLP6KPhCp+wOyMEMj4GLFmVPMHDVcNyHoHLXLExUv3rbIe8OorZYR6N7W2M3wXEpLiuNNdxv4c= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1683146451; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=MTUxCHqiQybvi1dONKBH/ejbHi2JtFT3ch8bE8x75TE=; b=ErSfx70DMfoRpv2ehK68tmVhTDgycwD0CC3Teg48eCEWu4RVrZLej39rbv+RBwM/+FQsqwsyFC7khra92A63wB8cFO4+5gjaIDLTyYUQO3R6vvzdTmSudzWMV7e3chTFUXxp+6DMIChgIlBiuh8fjlTsRAMTtKD4fOM/7ZHTIZw= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1683146451396314.44029667426094; Wed, 3 May 2023 13:40:51 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1puJGY-0007gJ-UO; Wed, 03 May 2023 16:39:58 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1puJGX-0007fi-Hn for qemu-devel@nongnu.org; Wed, 03 May 2023 16:39:57 -0400 Received: from smtp-out2.suse.de ([2001:67c:2178:6::1d]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1puJGV-0005sW-Ub for qemu-devel@nongnu.org; Wed, 03 May 2023 16:39:57 -0400 Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id 66928206E3; Wed, 3 May 2023 20:39:54 +0000 (UTC) Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id 9A48613584; Wed, 3 May 2023 20:39:52 +0000 (UTC) Received: from dovecot-director2.suse.de ([192.168.254.65]) by imap2.suse-dmz.suse.de with ESMTPSA id SPVUGJjGUmRofAAAMHmgww (envelope-from ); Wed, 03 May 2023 20:39:52 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1683146394; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=MTUxCHqiQybvi1dONKBH/ejbHi2JtFT3ch8bE8x75TE=; b=FzkmsQ4hU5o4+HBbO9IzufQJJOZ9thXO7weWBWlSFrpA1uTZxYiX4+C+sVcUtHW0YRlGOh dbSVctKt8pUZ752Yge1sYx1nFXi8pmh52rclQhi0wvcZWlIv+X6mAtoAWlL3gsa7e0TKHr /h1FJSd0L2RlgsbpKKEaOXv/CJ/wL3c= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1683146394; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=MTUxCHqiQybvi1dONKBH/ejbHi2JtFT3ch8bE8x75TE=; b=yNpB4UjSU5b3kbvfMQdGsoP7S81wgQugfLpPVok4bxsBD8tOA1WbM5eb5h0xieMytZ2bKn ylcXxktt+KnsklCw== From: Fabiano Rosas To: qemu-devel@nongnu.org Cc: =?UTF-8?q?Daniel=20P=20=2E=20Berrang=C3=A9?= , Paolo Bonzini , =?UTF-8?q?Marc-Andr=C3=A9=20Lureau?= , Peter Maydell , Thomas Huth Subject: [PATCH v4 1/2] meson: Pass -j option to sphinx Date: Wed, 3 May 2023 17:39:46 -0300 Message-Id: <20230503203947.3417-2-farosas@suse.de> X-Mailer: git-send-email 2.35.3 In-Reply-To: <20230503203947.3417-1-farosas@suse.de> References: <20230503203947.3417-1-farosas@suse.de> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=2001:67c:2178:6::1d; envelope-from=farosas@suse.de; helo=smtp-out2.suse.de X-Spam_score_int: -43 X-Spam_score: -4.4 X-Spam_bar: ---- X-Spam_report: (-4.4 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @suse.de) X-ZM-MESSAGEID: 1683146452224100001 Save a bit of build time by passing the number of jobs option to sphinx. We cannot use the -j option from make because meson does not support setting build time parameters for custom targets. Use nproc instead or the equivalent sphinx option "-j auto", if that is available (version >=3D1.7.0). Also make sure our plugins support parallelism and report it properly to sphinx. Particularly, implement the merge_domaindata method in DBusDomain that is used to merge in data from other subprocesses. Tested-by: Daniel P. Berrang=C3=A9 Signed-off-by: Fabiano Rosas --- docs/meson.build | 12 ++++++++++++ docs/sphinx/dbusdomain.py | 4 ++++ docs/sphinx/fakedbusdoc.py | 5 +++++ docs/sphinx/qmp_lexer.py | 5 +++++ 4 files changed, 26 insertions(+) diff --git a/docs/meson.build b/docs/meson.build index f220800e3e..6d0986579e 100644 --- a/docs/meson.build +++ b/docs/meson.build @@ -10,6 +10,18 @@ if sphinx_build.found() SPHINX_ARGS +=3D [ '-W', '-Dkerneldoc_werror=3D1' ] endif =20 + sphinx_version =3D run_command(SPHINX_ARGS + ['--version'], + check: true).stdout().split()[1] + if sphinx_version.version_compare('>=3D1.7.0') + SPHINX_ARGS +=3D ['-j', 'auto'] + else + nproc =3D find_program('nproc') + if nproc.found() + jobs =3D run_command(nproc, check: true).stdout() + SPHINX_ARGS +=3D ['-j', jobs] + endif + endif + # This is a bit awkward but works: create a trivial document and # try to run it with our configuration file (which enforces a # version requirement). This will fail if sphinx-build is too old. diff --git a/docs/sphinx/dbusdomain.py b/docs/sphinx/dbusdomain.py index 2ea95af623..9872fd5bf6 100644 --- a/docs/sphinx/dbusdomain.py +++ b/docs/sphinx/dbusdomain.py @@ -400,6 +400,10 @@ def get_objects(self) -> Iterator[Tuple[str, str, str,= str, str, int]]: for refname, obj in self.objects.items(): yield (refname, refname, obj.objtype, obj.docname, obj.node_id= , 1) =20 + def merge_domaindata(self, docnames, otherdata): + for name, obj in otherdata['objects'].items(): + if obj.docname in docnames: + self.data['objects'][name] =3D obj =20 def setup(app): app.add_domain(DBusDomain) diff --git a/docs/sphinx/fakedbusdoc.py b/docs/sphinx/fakedbusdoc.py index d2c5079046..2d2e6ef640 100644 --- a/docs/sphinx/fakedbusdoc.py +++ b/docs/sphinx/fakedbusdoc.py @@ -23,3 +23,8 @@ def run(self): def setup(app: Sphinx) -> Dict[str, Any]: """Register a fake dbus-doc directive with Sphinx""" app.add_directive("dbus-doc", FakeDBusDocDirective) + + return dict( + parallel_read_safe =3D True, + parallel_write_safe =3D True + ) diff --git a/docs/sphinx/qmp_lexer.py b/docs/sphinx/qmp_lexer.py index f7e4c0e198..a59de8a079 100644 --- a/docs/sphinx/qmp_lexer.py +++ b/docs/sphinx/qmp_lexer.py @@ -41,3 +41,8 @@ def setup(sphinx): sphinx.add_lexer('QMP', QMPExampleLexer) except errors.VersionRequirementError: sphinx.add_lexer('QMP', QMPExampleLexer()) + + return dict( + parallel_read_safe =3D True, + parallel_write_safe =3D True + ) --=20 2.35.3 From nobody Tue May 14 01:29:24 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=suse.de ARC-Seal: i=1; a=rsa-sha256; t=1683146452; cv=none; d=zohomail.com; s=zohoarc; b=UtUCCgkWZTbAqQMUsdZy/W3piHGG2N3LsfhDa8BY/GvlA6nIKMLX1C3oHkOjaieODcA9q1URwHwyjmYuWf69RJpi5iUEmP59LfZg17V36cMsKTzlTHLykdw5j1GK5t5/KggIDpqV4higk4odnFhjvoZSDwkEbivEg6VJBW0WPsw= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1683146452; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=urXhDKl2MolY1NPxpyuZ/N/nDOlLqUMu1hjJCx4bfJE=; b=HYn5EmYJFdNCjx8t2gbhAzZXYH3iOrzIu6Xpouz4kELAq6R1q0AWgWM4p6NgJ8Ufw7gN8x1+zcHGPVKfPmieMsBHQwPfdOCTopYUWKklyNRcA4qU68AgznAoD07Lnai49Ea9p7TS13XXcZWGAi021kEm6jCZh6NO6qh1aotXUP0= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1683146452342313.91189288380167; Wed, 3 May 2023 13:40:52 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1puJGa-0007gl-NB; Wed, 03 May 2023 16:40:00 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1puJGZ-0007gY-Gp for qemu-devel@nongnu.org; Wed, 03 May 2023 16:39:59 -0400 Received: from smtp-out1.suse.de ([195.135.220.28]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1puJGX-0005sl-S7 for qemu-devel@nongnu.org; Wed, 03 May 2023 16:39:59 -0400 Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id A4D2A22CC8; Wed, 3 May 2023 20:39:56 +0000 (UTC) Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id D462813584; Wed, 3 May 2023 20:39:54 +0000 (UTC) Received: from dovecot-director2.suse.de ([192.168.254.65]) by imap2.suse-dmz.suse.de with ESMTPSA id oMyyJprGUmRofAAAMHmgww (envelope-from ); Wed, 03 May 2023 20:39:54 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1683146396; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=urXhDKl2MolY1NPxpyuZ/N/nDOlLqUMu1hjJCx4bfJE=; b=IT1nfuHlRDMs4HQnirS+VnfV8yCJRmXC6qOhk2DS0zhdumK6lvjBDCfuYCiGPndLJKJPYf jqo78TfAisku4FHP+x5RASf2OcGlrDAEDMMBbO7tslwYpXe05GeOHutzAqprlOzXAmorNO 31n+FlUzM6CGtrl65P69CcJ1dTnEn3I= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1683146396; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=urXhDKl2MolY1NPxpyuZ/N/nDOlLqUMu1hjJCx4bfJE=; b=aO5mFcSxMstd71d+RPKkMntztM0TgZ+dDBw2CetolpZbzx9lUqa0ylj9ga5BsCRlMCIQKm GTgM4yrM1n79cbCg== From: Fabiano Rosas To: qemu-devel@nongnu.org Cc: =?UTF-8?q?Daniel=20P=20=2E=20Berrang=C3=A9?= , Paolo Bonzini , =?UTF-8?q?Marc-Andr=C3=A9=20Lureau?= , Peter Maydell , Thomas Huth Subject: [PATCH v4 2/2] meson: Deserialize the man pages and html builds Date: Wed, 3 May 2023 17:39:47 -0300 Message-Id: <20230503203947.3417-3-farosas@suse.de> X-Mailer: git-send-email 2.35.3 In-Reply-To: <20230503203947.3417-1-farosas@suse.de> References: <20230503203947.3417-1-farosas@suse.de> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=195.135.220.28; envelope-from=farosas@suse.de; helo=smtp-out1.suse.de X-Spam_score_int: -43 X-Spam_score: -4.4 X-Spam_bar: ---- X-Spam_report: (-4.4 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @suse.de) X-ZM-MESSAGEID: 1683146454561100011 Content-Type: text/plain; charset="utf-8" For the documentation builds (man pages & manual), we let Sphinx decide when to rebuild and use a depfile to know when to trigger the make target. We currently use a trick of having the man pages custom_target take as input the html pages custom_target object, which causes both targets to be executed if one of the dependencies has changed. However, having this at the custom_target level means that the two builds are effectively serialized. We can eliminate the dependency between the targets by adding a second depfile for the man pages build, allowing them to be parallelized by ninja while keeping sphinx in charge of deciding when to rebuild. Since they can now run in parallel, separate the Sphinx cache directory of the two builds. We need this not only for data consistency but also because Sphinx writes builder-dependent environment information to the cache directory (see notes under smartquotes_excludes in sphinx docs [1]). Note that after this patch the commands `make man` and `make html` only build the specified target. To keep the old behavior of building both targets, use `make man html` or `make sphinxdocs`. 1- https://www.sphinx-doc.org/en/master/usage/configuration.html Signed-off-by: Fabiano Rosas --- docs/meson.build | 36 ++++++++++++++++++++---------------- 1 file changed, 20 insertions(+), 16 deletions(-) diff --git a/docs/meson.build b/docs/meson.build index 6d0986579e..858e737431 100644 --- a/docs/meson.build +++ b/docs/meson.build @@ -42,7 +42,9 @@ if sphinx_build.found() endif =20 if build_docs - SPHINX_ARGS +=3D ['-Dversion=3D' + meson.project_version(), '-Drelease= =3D' + get_option('pkgversion')] + SPHINX_ARGS +=3D ['-Dversion=3D' + meson.project_version(), + '-Drelease=3D' + get_option('pkgversion'), + '-Ddepfile=3D@DEPFILE@', '-Ddepfile_stamp=3D@OUTPUT0@'] =20 man_pages =3D { 'qemu-ga.8': (have_ga ? 'man8' : ''), @@ -61,41 +63,43 @@ if build_docs } =20 sphinxdocs =3D [] - sphinxmans =3D [] =20 private_dir =3D meson.current_build_dir() / 'manual.p' output_dir =3D meson.current_build_dir() / 'manual' input_dir =3D meson.current_source_dir() =20 - this_manual =3D custom_target('QEMU manual', + manual =3D custom_target('QEMU manual', build_by_default: build_docs, - output: 'docs.stamp', + output: 'manual.stamp', input: files('conf.py'), - depfile: 'docs.d', - command: [SPHINX_ARGS, '-Ddepfile=3D@DEPFILE@', - '-Ddepfile_stamp=3D@OUTPUT0@', - '-b', 'html', '-d', private_dir, + depfile: 'manual.dep', + command: [SPHINX_ARGS, '-b', 'html', '-d', private_dir, input_dir, output_dir]) - sphinxdocs +=3D this_manual + sphinxdocs +=3D manual install_subdir(output_dir, install_dir: qemu_docdir, strip_directory: tr= ue) =20 - these_man_pages =3D [] - install_dirs =3D [] + man_private_dir =3D meson.current_build_dir() / 'man.p' + # man.stamp is not installed + these_man_pages =3D ['man.stamp'] + install_dirs =3D [false] + foreach page, section : man_pages these_man_pages +=3D page install_dirs +=3D section =3D=3D '' ? false : get_option('mandir') / s= ection endforeach =20 - sphinxmans +=3D custom_target('QEMU man pages', + + man_pages =3D custom_target('QEMU man pages', build_by_default: build_docs, output: these_man_pages, - input: this_manual, + depfile: 'man.dep', install: build_docs, install_dir: install_dirs, - command: [SPHINX_ARGS, '-b', 'man', '-d', pr= ivate_dir, + command: [SPHINX_ARGS, '-b', 'man', '-d', ma= n_private_dir, input_dir, meson.current_build_dir= ()]) + sphinxdocs +=3D man_pages =20 alias_target('sphinxdocs', sphinxdocs) - alias_target('html', sphinxdocs) - alias_target('man', sphinxmans) + alias_target('html', manual) + alias_target('man', man_pages) endif --=20 2.35.3