This patchset converts the following documentation to rST format:
 * qemu-img
 * qemu-trace-stap
 * virtfs-proxy-helper

(That means everything in step 3 in the plan:
https://wiki.qemu.org/Features/Documentation#3:_Convert_things_which_are_mostly_standalone_manpages
will be done except for qemu-cpu-models.texi. That
should be a straightforward conversion but I haven't
touched it yet because I know there's an on-list patch
that updates the texi and wanted to avoid a conflict.)

The patchset includes a new Sphinx extension which handles parsing
the .hx files which provide documentation fragments for the qemu-img
manual.

Changes from v1 to v2:
 * rebased on master, since the qemu-nbd conversion has now
   gone in
 * the patches at the end to convert qemu-trace-stap and
   virtfs-proxy-helper are new
 * new patch at the start of the series which fixes a
   bug in our makefiles where we could try to invoke
   Sphinx twice in parallel on the same doctree (which
   causes it to crash, as well as being unnecessary work)
 * fixed the import line for ExtensionError, so this
   should now work with Sphinx 1.8

I've assigned manpages to interop/ or system/ according
to the structure set out in the wiki page above. We should
have a discussion about whether some of these should go
in a new tools/ manual or not (I'll start a separate
thread for that), but it's easy enough to move them
later if we need to.

The general approach follows the outline in the email I
sent the other day:
https://lists.gnu.org/archive/html/qemu-devel/2020-01/msg03786.html

The new Sphinx extension implements the hxtool-doc::
directive, which indicates where the assembled rST
document fragments should be inserted into the manual.
qemu-img-cmds.hx doesn't use the DEFHEADING or ARCHHEADING
directives, but the extension implements them (tested
with some local modifications to the .hx file to check
that they do the right thing).

As noted in the commit message for the qemu-img.texi conversion,
I have not attempted to tackle any of the muddle in the
current documentation structure or the repetition between
the manual document, the fragments in the .hx file and
the C code; this is a "simplest thing that works"
like-for-like conversion.

Another deliberate omission is that I have not attempted
to get links between our various Sphinx manuals (system,
interop, etc) working yet, as this is not totally trivial
and the odd minor missed hyperlink doesn't seem to me
to be a deal-breaker.

Sorry about the size of the main 'convert qemu-img'
patch, but it's unavoidable when converting a big
document between formats.

thanks
-- PMM

Peter Maydell (8):
  Makefile: Ensure we don't run Sphinx in parallel for manpages
  hxtool: Support SRST/ERST directives
  docs/sphinx: Add new hxtool Sphinx extension
  qemu-img-cmds.hx: Add rST documentation fragments
  qemu-img: Convert invocation documentation to rST
  qemu-img-cmds.hx: Remove texinfo document fragments
  scripts/qemu-trace-stap: Convert documentation to rST
  virtfs-proxy-helper: Convert documentation to rST

 Makefile                             |  46 +-
 MAINTAINERS                          |   3 +
 docs/conf.py                         |   3 +-
 docs/interop/conf.py                 |   9 +-
 docs/interop/index.rst               |   3 +
 docs/interop/qemu-img.rst            | 822 +++++++++++++++++++++++++++
 docs/interop/qemu-trace-stap.rst     | 124 ++++
 docs/interop/virtfs-proxy-helper.rst |  72 +++
 docs/sphinx/hxtool.py                | 210 +++++++
 fsdev/virtfs-proxy-helper.texi       |  63 --
 qemu-doc.texi                        |  10 +-
 qemu-img-cmds.hx                     |  99 ++--
 qemu-img.texi                        | 795 --------------------------
 rules.mak                            |  36 ++
 scripts/hxtool                       |  33 +-
 scripts/qemu-trace-stap.texi         | 140 -----
 16 files changed, 1383 insertions(+), 1085 deletions(-)
 create mode 100644 docs/interop/qemu-img.rst
 create mode 100644 docs/interop/qemu-trace-stap.rst
 create mode 100644 docs/interop/virtfs-proxy-helper.rst
 create mode 100644 docs/sphinx/hxtool.py
 delete mode 100644 fsdev/virtfs-proxy-helper.texi
 delete mode 100644 qemu-img.texi
 delete mode 100644 scripts/qemu-trace-stap.texi

-- 
2.20.1

On Fri, 24 Jan 2020 at 16:26, Peter Maydell <peter.maydell@linaro.org> wrote:
>
> This patchset converts the following documentation to rST format:
>  * qemu-img
>  * qemu-trace-stap
>  * virtfs-proxy-helper
>
> (That means everything in step 3 in the plan:
> https://wiki.qemu.org/Features/Documentation#3:_Convert_things_which_are_mostly_standalone_manpages
> will be done except for qemu-cpu-models.texi. That
> should be a straightforward conversion but I haven't
> touched it yet because I know there's an on-list patch
> that updates the texi and wanted to avoid a conflict.)
>
> The patchset includes a new Sphinx extension which handles parsing
> the .hx files which provide documentation fragments for the qemu-img
> manual.
>

I'm going to send out a pullreq today with this in.
I'm squashing in these trivial fixups that deal with a
conflict with commit cdd267749a3ab787e8b, which added
a new qemu-img option and its documentation:

--- a/docs/interop/qemu-img.rst
+++ b/docs/interop/qemu-img.rst
@@ -247,7 +247,7 @@ Command description:
   Amends the image format specific *OPTIONS* for the image file
   *FILENAME*. Not all file formats support this operation.

-.. option:: bench [-c COUNT] [-d DEPTH] [-f FMT]
[--flush-interval=FLUSH_INTERVAL] [-n] [--no-drain] [-o OFFSET]
[--pattern=PATTERN] [-q] [-s BUFFER_SIZE] [-S STEP_SIZE] [-t CACHE]
[-w] [-U] FILENAME
+.. option:: bench [-c COUNT] [-d DEPTH] [-f FMT]
[--flush-interval=FLUSH_INTERVAL] [-n] [-i AIO] [--no-drain] [-o
OFFSET] [--pattern=PATTERN] [-q] [-s BUFFER_SIZE] [-S STEP_SIZE] [-t
CACHE] [-w] [-U] FILENAME

   Run a simple sequential I/O benchmark on the specified image. If ``-w`` is
   specified, a write test is performed, otherwise a read test is performed.
@@ -268,6 +268,9 @@ Command description:
   Linux, this option only works if ``-t none`` or ``-t directsync`` is
   specified as well.

+  if ``-i`` is specified, *AIO* option can be used to specify different
+  AIO backends: ``threads``, ``native`` or ``io_uring``.
+
   For write tests, by default a buffer filled with zeros is written.
This can be
   overridden with a pattern byte specified by *PATTERN*.


--- a/qemu-img-cmds.hx
+++ b/qemu-img-cmds.hx
@@ -28,7 +28,7 @@ STEXI
 @item bench [-c @var{count}] [-d @var{depth}] [-f @var{fmt}]
[--flush-interval=@var{flush_interval}] [-n] [--no-drain] [-o
@var{offset}] [--pattern=@var{pattern}] [-q] [-s @var{buffer_size}]
[-S @var{step_size}] [-t @var{cache}] [-i @var{aio}] [-w] [-U]
@var{filename}
 ETEXI
 SRST
-.. option:: bench [-c COUNT] [-d DEPTH] [-f FMT]
[--flush-interval=FLUSH_INTERVAL] [-n] [--no-drain] [-o OFFSET]
[--pattern=PATTERN] [-q] [-s BUFFER_SIZE] [-S STEP_SIZE] [-t CACHE]
[-w] [-U] FILENAME
+.. option:: bench [-c COUNT] [-d DEPTH] [-f FMT]
[--flush-interval=FLUSH_INTERVAL] [-n] [--no-drain] [-o OFFSET]
[--pattern=PATTERN] [-q] [-s BUFFER_SIZE] [-S STEP_SIZE] [-t CACHE]
[-i AIO] [-w] [-U] FILENAME
 ERST
 DEF("check", img_check,
     "check [--object objectdef] [--image-opts] [-q] [-f fmt]
[--output=ofmt] [-r [leaks | all]] [-T src_cache] [-U] filename")

thanks
-- PMM

I know it's only a week, but could I ping for review on this?
(statements of the form "I do want to review but need more
time" also welcome.) Given the potential for conflicts with
other changes that touch the docs, plus that there's other
patchsets which depend on this one, it would be nice to put
it into the tree sooner rather than later.

Still in need of review: patches 1 (makefile magic),
5 (the qemu-img conversion itself), 7 (qemu-trace-stap).

thanks
-- PMM

On Fri, 24 Jan 2020 at 16:26, Peter Maydell <peter.maydell@linaro.org> wrote:
>
> This patchset converts the following documentation to rST format:
>  * qemu-img
>  * qemu-trace-stap
>  * virtfs-proxy-helper
>
> (That means everything in step 3 in the plan:
> https://wiki.qemu.org/Features/Documentation#3:_Convert_things_which_are_mostly_standalone_manpages
> will be done except for qemu-cpu-models.texi. That
> should be a straightforward conversion but I haven't
> touched it yet because I know there's an on-list patch
> that updates the texi and wanted to avoid a conflict.)
>
> The patchset includes a new Sphinx extension which handles parsing
> the .hx files which provide documentation fragments for the qemu-img
> manual.
>
> Changes from v1 to v2:
>  * rebased on master, since the qemu-nbd conversion has now
>    gone in
>  * the patches at the end to convert qemu-trace-stap and
>    virtfs-proxy-helper are new
>  * new patch at the start of the series which fixes a
>    bug in our makefiles where we could try to invoke
>    Sphinx twice in parallel on the same doctree (which
>    causes it to crash, as well as being unnecessary work)
>  * fixed the import line for ExtensionError, so this
>    should now work with Sphinx 1.8
>
> I've assigned manpages to interop/ or system/ according
> to the structure set out in the wiki page above. We should
> have a discussion about whether some of these should go
> in a new tools/ manual or not (I'll start a separate
> thread for that), but it's easy enough to move them
> later if we need to.
>
> The general approach follows the outline in the email I
> sent the other day:
> https://lists.gnu.org/archive/html/qemu-devel/2020-01/msg03786.html
>
> The new Sphinx extension implements the hxtool-doc::
> directive, which indicates where the assembled rST
> document fragments should be inserted into the manual.
> qemu-img-cmds.hx doesn't use the DEFHEADING or ARCHHEADING
> directives, but the extension implements them (tested
> with some local modifications to the .hx file to check
> that they do the right thing).
>
> As noted in the commit message for the qemu-img.texi conversion,
> I have not attempted to tackle any of the muddle in the
> current documentation structure or the repetition between
> the manual document, the fragments in the .hx file and
> the C code; this is a "simplest thing that works"
> like-for-like conversion.
>
> Another deliberate omission is that I have not attempted
> to get links between our various Sphinx manuals (system,
> interop, etc) working yet, as this is not totally trivial
> and the odd minor missed hyperlink doesn't seem to me
> to be a deal-breaker.
>
> Sorry about the size of the main 'convert qemu-img'
> patch, but it's unavoidable when converting a big
> document between formats.
>
> thanks
> -- PMM
>
> Peter Maydell (8):
>   Makefile: Ensure we don't run Sphinx in parallel for manpages
>   hxtool: Support SRST/ERST directives
>   docs/sphinx: Add new hxtool Sphinx extension
>   qemu-img-cmds.hx: Add rST documentation fragments
>   qemu-img: Convert invocation documentation to rST
>   qemu-img-cmds.hx: Remove texinfo document fragments
>   scripts/qemu-trace-stap: Convert documentation to rST
>   virtfs-proxy-helper: Convert documentation to rST
>
>  Makefile                             |  46 +-
>  MAINTAINERS                          |   3 +
>  docs/conf.py                         |   3 +-
>  docs/interop/conf.py                 |   9 +-
>  docs/interop/index.rst               |   3 +
>  docs/interop/qemu-img.rst            | 822 +++++++++++++++++++++++++++
>  docs/interop/qemu-trace-stap.rst     | 124 ++++
>  docs/interop/virtfs-proxy-helper.rst |  72 +++
>  docs/sphinx/hxtool.py                | 210 +++++++
>  fsdev/virtfs-proxy-helper.texi       |  63 --
>  qemu-doc.texi                        |  10 +-
>  qemu-img-cmds.hx                     |  99 ++--
>  qemu-img.texi                        | 795 --------------------------
>  rules.mak                            |  36 ++
>  scripts/hxtool                       |  33 +-
>  scripts/qemu-trace-stap.texi         | 140 -----
>  16 files changed, 1383 insertions(+), 1085 deletions(-)
>  create mode 100644 docs/interop/qemu-img.rst
>  create mode 100644 docs/interop/qemu-trace-stap.rst
>  create mode 100644 docs/interop/virtfs-proxy-helper.rst
>  create mode 100644 docs/sphinx/hxtool.py
>  delete mode 100644 fsdev/virtfs-proxy-helper.texi
>  delete mode 100644 qemu-img.texi
>  delete mode 100644 scripts/qemu-trace-stap.texi
>
> --
> 2.20.1