From nobody Mon Feb 9 18:18:53 2026 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=linaro.org ARC-Seal: i=1; a=rsa-sha256; t=1579883268; cv=none; d=zohomail.com; s=zohoarc; b=HU3FXTfybGqlWB48HREJPcxYi8fGevy03uIcALy89O7ZGw1ZC2ioGRvR5pUa9wcTwjj4TNvmtWabY5GJUOfzgdO4MyilAIJ4969I4Dzs/oCz+5QzXNhLNFtQsANF54/DrJXq8g5X35GB1mUHvZMkuc99vPK1USrPDg6QYEJb4yU= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1579883268; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=oOLCLALD3GcgJUBOZ9tz3obHiRR8ETIrIGcxOBvtbNk=; b=NKf+ZRoW3vJqLT1zT0u5BoNy14z4+JO/uWZs+8X92o07l62PWhS0g7XIObg4LkgCi4vc1cxzsTRRlvZ9dbP/fiOZhFMsVhocogmO4LBDi093EMqOcVOwY3mVhX2JTD/kEVxQ69Ft18ppxWIJ3Tm2mu2IYxmn3xgRnDvZWbkKn48= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1579883268887617.0589471207346; Fri, 24 Jan 2020 08:27:48 -0800 (PST) Received: from localhost ([::1]:44430 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iv1oB-0004tp-Pt for importer@patchew.org; Fri, 24 Jan 2020 11:27:47 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:37906) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iv1mh-0002SU-Ck for qemu-devel@nongnu.org; Fri, 24 Jan 2020 11:26:18 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iv1mf-0002oD-UP for qemu-devel@nongnu.org; Fri, 24 Jan 2020 11:26:15 -0500 Received: from mail-wm1-x32f.google.com ([2a00:1450:4864:20::32f]:52077) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1iv1mf-0002ms-On for qemu-devel@nongnu.org; Fri, 24 Jan 2020 11:26:13 -0500 Received: by mail-wm1-x32f.google.com with SMTP id t23so56443wmi.1 for ; Fri, 24 Jan 2020 08:26:13 -0800 (PST) Received: from orth.archaic.org.uk (orth.archaic.org.uk. [81.2.115.148]) by smtp.gmail.com with ESMTPSA id q3sm7054998wmj.38.2020.01.24.08.26.09 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 24 Jan 2020 08:26:10 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=oOLCLALD3GcgJUBOZ9tz3obHiRR8ETIrIGcxOBvtbNk=; b=OBbo5yX7o6dwaoqXuL1Is8LOx6HZBSkcU0PpObp/WwXEi0/80kPJnas0UZd9jLZzHS JL4TA6bXVMCw6MMOLZq3qSXeX3igxYgGOJyfKE1KixG3LQf3nHBha71RTHSFz3wcEBnF rWe6l74iNmeCjuLfIc/jlMPQectzaHKORod70eh7XYrSh7htONcZKxV+90sXm/PrtF6i Wk87+Y61+cMJeOoC/83aBj/eBl6lyZT4Nw63H1Sca5xITXGZ5Jm0erivwcomMwpj1R9c uIqTDHWpVcYfkghknMlwRushYZNinHtAGW+5/oZNs5FWOIzF6smTgYaSy5rllHp35mXd aFtw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=oOLCLALD3GcgJUBOZ9tz3obHiRR8ETIrIGcxOBvtbNk=; b=nw8JTdAgKxEDShTIBT/jMaP6qapj8qptAcx24QOUBGzylNMix7YBmpDetBBAlZYaSh z1zzhEiSoNhrTf2JU0Vm01lBLh78IRij3STngZ3suCvnQBSdITg5tcitr/S4jmYVAejs TjcNOpBD/sHenNxnUFGFoPdTndnGHG86w2TMoQVxpHhtlZaLgqDw4RIq97roNtxdxY79 5QWds5yW8Jbg/IGhi4TPMrGd1AB/mtX9S7n14hMWySjpPL/FXb4a0E3C8mu1L5go5aeq zUPNvvAWe/t0exqp5sXHT3p/blVA30tNpOhJkiH2c4CVKNMI8kzRxQXQK7dd0RvLuxUu TkQA== X-Gm-Message-State: APjAAAVwgSY281CUynswnRXmKQZ4D3Vs8HX8giwowA2RxliG7W2VeMpa a8QZkd/Qu/2fuqFsF3H0YpLHdX005rlEGA== X-Google-Smtp-Source: APXvYqyoARovtMX5fFiulhMbDetgUj9B0CwlmuOP9vflUWZdCLIItAd1yVhRGslZjznXYuqslcQrJg== X-Received: by 2002:a1c:a796:: with SMTP id q144mr70633wme.6.1579883170885; Fri, 24 Jan 2020 08:26:10 -0800 (PST) From: Peter Maydell To: qemu-devel@nongnu.org Subject: [PATCH v2 1/8] Makefile: Ensure we don't run Sphinx in parallel for manpages Date: Fri, 24 Jan 2020 16:25:59 +0000 Message-Id: <20200124162606.8787-2-peter.maydell@linaro.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20200124162606.8787-1-peter.maydell@linaro.org> References: <20200124162606.8787-1-peter.maydell@linaro.org> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2a00:1450:4864:20::32f X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , qemu-block@nongnu.org, Markus Armbruster , Richard Henderson , Greg Kurz , Max Reitz , Stefan Hajnoczi , John Snow Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) Content-Type: text/plain; charset="utf-8" Sphinx will corrupt its doctree cache if we run two copies of it in parallel. In commit 6bda415c10d966c8d3 we worked around this by having separate doctrees for 'html' vs 'manpage' runs. However now that we have more than one manpage produced from a single manual we can run into this again when trying to produce the two manpages. Use the trick described in 'Atomic Rules in GNU Make' https://www.cmcrossroads.com/article/atomic-rules-gnu-make to ensure that we only run the Sphinx manpage builder once for each manual, even if we're producing several manpages. This fixes doctree corruption in parallel builds and also avoids pointlessly running Sphinx more often than we need to. (In GNU Make 4.3 there is builtin support for this, via the "&:" syntax, but we can't wait for that to be available in all the distros we support...) The generic "one invocation for multiple output files" machinery is provided as a macro named 'atomic' in rules.mak; we then wrap this in a more specific macro for defining the rule and dependencies for the manpages in a Sphinx manual, to avoid excessive repetition. Signed-off-by: Peter Maydell Reviewed-by: Alex Benn=C3=A9e Tested-by: Alex Benn=C3=A9e --- Makefile | 17 ++++++++++------- rules.mak | 36 ++++++++++++++++++++++++++++++++++++ 2 files changed, 46 insertions(+), 7 deletions(-) diff --git a/Makefile b/Makefile index 04c77d3b962..9b7ff1dc82f 100644 --- a/Makefile +++ b/Makefile @@ -1028,6 +1028,14 @@ build-manual =3D $(call quiet-command,CONFDIR=3D"$(q= emu_confdir)" sphinx-build $(if manual-deps =3D $(wildcard $(SRC_PATH)/docs/$1/*.rst) \ $(wildcard $(SRC_PATH)/docs/$1/*.rst.inc) \ $(SRC_PATH)/docs/$1/conf.py $(SRC_PATH)/docs/conf.py +# Macro to write out the rule and dependencies for building manpages +# Usage: $(call define-manpage-rule,manualname,manpage1 manpage2...[,extra= deps]) +# 'extradeps' is optional, and specifies extra files (eg .hx files) that +# the manual page depends on. +define define-manpage-rule +$(call atomic,$(foreach manpage,$2,$(MANUAL_BUILDDIR)/$1/$(manpage)),$(cal= l manual-deps,$1) $3) + $(call build-manual,$1,man) +endef =20 $(MANUAL_BUILDDIR)/devel/index.html: $(call manual-deps,devel) $(call build-manual,devel,html) @@ -1041,14 +1049,9 @@ $(MANUAL_BUILDDIR)/specs/index.html: $(call manual-d= eps,specs) $(MANUAL_BUILDDIR)/system/index.html: $(call manual-deps,system) $(call build-manual,system,html) =20 -$(MANUAL_BUILDDIR)/interop/qemu-ga.8: $(call manual-deps,interop) - $(call build-manual,interop,man) +$(call define-manpage-rule,interop,qemu-ga.8 qemu-nbd.8) =20 -$(MANUAL_BUILDDIR)/interop/qemu-nbd.8: $(call manual-deps,interop) - $(call build-manual,interop,man) - -$(MANUAL_BUILDDIR)/system/qemu-block-drivers.7: $(call manual-deps,system) - $(call build-manual,system,man) +$(call define-manpage-rule,system,qemu-block-drivers.7) =20 $(MANUAL_BUILDDIR)/index.html: $(SRC_PATH)/docs/index.html.in qemu-version= .h @mkdir -p "$(MANUAL_BUILDDIR)" diff --git a/rules.mak b/rules.mak index 967295dd2b6..50f6776f529 100644 --- a/rules.mak +++ b/rules.mak @@ -399,3 +399,39 @@ GEN_SUBST =3D $(call quiet-command, \ =20 %.json: %.json.in $(call GEN_SUBST) + +# Support for building multiple output files by atomically executing +# a single rule which depends on several input files (so the rule +# will be executed exactly once, not once per output file, and +# not multiple times in parallel.) For more explanation see: +# https://www.cmcrossroads.com/article/atomic-rules-gnu-make + +# Given a space-separated list of filenames, create the name of +# a 'sentinel' file to use to indicate that they have been built. +# We use fixed text on the end to avoid accidentally triggering +# automatic pattern rules, and . on the start to make the file +# not show up in ls output. +sentinel =3D .$(subst $(SPACE),_,$(subst /,_,$1)).sentinel. + +# Define an atomic rule that builds multiple outputs from multiple inputs. +# To use: +# $(call atomic,out1 out2 ...,in1 in2 ...) +# rule to do the operation +# +# Make 4.3 will have native support for this, and you would be able +# to instead write: +# out1 out2 ... &: in1 in2 ... +# rule to do the operation +# +# The way this works is that it creates a make rule +# "out1 out2 ... : sentinel-file ; @:" which says that the sentinel +# depends on the dependencies, and the rule to do that is "do nothing". +# Then we have a rule +# "sentinel-file : in1 in2 ..." +# whose commands start with "touch sentinel-file" and then continue +# with the rule text provided by the user of this 'atomic' function. +# The foreach... is there to delete the sentinel file if any of the +# output files don't exist, so that we correctly rebuild in that situation. +atomic =3D $(eval $1: $(call sentinel,$1) ; @:) \ + $(call sentinel,$1) : $2 ; @touch $$@ \ + $(foreach t,$1,$(if $(wildcard $t),,$(shell rm -f $(call sentinel= ,$1)))) --=20 2.20.1