We have a very frequent pattern of creating a coroutine from a function
with several arguments:
- create a structure to pack parameters
- create _entry function to call original function taking parameters
from struct
- do different magic to handle completion: set ret to NOT_DONE or
EINPROGRESS or use separate bool field
- fill the struct and create coroutine from _entry function with this
struct as a parameter
- do coroutine enter and BDRV_POLL_WHILE loop
Let's reduce code duplication by generating coroutine wrappers.
This patch adds scripts/block-coroutine-wrapper.py together with some
friends, which will generate functions with declared prototypes marked
by the 'generated_co_wrapper' specifier.
The usage of new code generation is as follows:
1. define the coroutine function somewhere
int coroutine_fn bdrv_co_NAME(...) {...}
2. declare in some header file
int generated_co_wrapper bdrv_NAME(...);
with same list of parameters (generated_co_wrapper is
defined in "include/block/block.h").
3. Make sure the block_gen_c delaration in block/meson.build
mentions the file with your marker function.
Still, no function is now marked, this work is for the following
commit.
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
---
docs/devel/block-coroutine-wrapper.rst | 54 +++++++
docs/devel/index.rst | 1 +
block/block-gen.h | 49 +++++++
include/block/block.h | 10 ++
block/meson.build | 8 ++
scripts/block-coroutine-wrapper.py | 188 +++++++++++++++++++++++++
6 files changed, 310 insertions(+)
create mode 100644 docs/devel/block-coroutine-wrapper.rst
create mode 100644 block/block-gen.h
create mode 100644 scripts/block-coroutine-wrapper.py
diff --git a/docs/devel/block-coroutine-wrapper.rst b/docs/devel/block-coroutine-wrapper.rst
new file mode 100644
index 0000000000..d09fff2cc5
--- /dev/null
+++ b/docs/devel/block-coroutine-wrapper.rst
@@ -0,0 +1,54 @@
+=======================
+block-coroutine-wrapper
+=======================
+
+A lot of functions in QEMU block layer (see ``block/*``) can only be
+called in coroutine context. Such functions are normally marked by the
+coroutine_fn specifier. Still, sometimes we need to call them from
+non-coroutine context; for this we need to start a coroutine, run the
+needed function from it and wait for coroutine finish in
+BDRV_POLL_WHILE() loop. To run a coroutine we need a function with one
+void* argument. So for each coroutine_fn function which needs a
+non-coroutine interface, we should define a structure to pack the
+parameters, define a separate function to unpack the parameters and
+call the original function and finally define a new interface function
+with same list of arguments as original one, which will pack the
+parameters into a struct, create a coroutine, run it and wait in
+BDRV_POLL_WHILE() loop. It's boring to create such wrappers by hand,
+so we have a script to generate them.
+
+Usage
+=====
+
+Assume we have defined the ``coroutine_fn`` function
+``bdrv_co_foo(<some args>)`` and need a non-coroutine interface for it,
+called ``bdrv_foo(<same args>)``. In this case the script can help. To
+trigger the generation:
+
+1. You need ``bdrv_foo`` declaration somewhere (for example, in
+ ``block/coroutines.h``) with the ``generated_co_wrapper`` mark,
+ like this:
+
+.. code-block:: c
+
+ int generated_co_wrapper bdrv_foo(<some args>);
+
+2. You need to feed this declaration to block-coroutine-wrapper script.
+ For this, add the .h (or .c) file with the declaration to the
+ ``input: files(...)`` list of ``block_gen_c`` target declaration in
+ ``block/meson.build``
+
+You are done. During the build, coroutine wrappers will be generated in
+``<BUILD_DIR>/block/block-gen.c``.
+
+Links
+=====
+
+1. The script location is ``scripts/block-coroutine-wrapper.py``.
+
+2. Generic place for private ``generated_co_wrapper`` declarations is
+ ``block/coroutines.h``, for public declarations:
+ ``include/block/block.h``
+
+3. The core API of generated coroutine wrappers is placed in
+ (not generated) ``block/block-gen.h``
diff --git a/docs/devel/index.rst b/docs/devel/index.rst
index 04773ce076..cb0abe1e69 100644
--- a/docs/devel/index.rst
+++ b/docs/devel/index.rst
@@ -31,3 +31,4 @@ Contents:
reset
s390-dasd-ipl
clocks
+ block-coroutine-wrapper
diff --git a/block/block-gen.h b/block/block-gen.h
new file mode 100644
index 0000000000..f80cf4897d
--- /dev/null
+++ b/block/block-gen.h
@@ -0,0 +1,49 @@
+/*
+ * Block coroutine wrapping core, used by auto-generated block/block-gen.c
+ *
+ * Copyright (c) 2003 Fabrice Bellard
+ * Copyright (c) 2020 Virtuozzo International GmbH
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a copy
+ * of this software and associated documentation files (the "Software"), to deal
+ * in the Software without restriction, including without limitation the rights
+ * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+ * copies of the Software, and to permit persons to whom the Software is
+ * furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+ * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
+ * THE SOFTWARE.
+ */
+
+#ifndef BLOCK_BLOCK_GEN_H
+#define BLOCK_BLOCK_GEN_H
+
+#include "block/block_int.h"
+
+/* Base structure for argument packing structures */
+typedef struct BdrvPollCo {
+ BlockDriverState *bs;
+ bool in_progress;
+ int ret;
+ Coroutine *co; /* Keep pointer here for debugging */
+} BdrvPollCo;
+
+static inline int bdrv_poll_co(BdrvPollCo *s)
+{
+ assert(!qemu_in_coroutine());
+
+ bdrv_coroutine_enter(s->bs, s->co);
+ BDRV_POLL_WHILE(s->bs, s->in_progress);
+
+ return s->ret;
+}
+
+#endif /* BLOCK_BLOCK_GEN_H */
diff --git a/include/block/block.h b/include/block/block.h
index 81d591dd4c..0f0ddc51b4 100644
--- a/include/block/block.h
+++ b/include/block/block.h
@@ -10,6 +10,16 @@
#include "block/blockjob.h"
#include "qemu/hbitmap.h"
+/*
+ * generated_co_wrapper
+ *
+ * Function specifier, which does nothing but mark functions to be
+ * generated by scripts/block-coroutine-wrapper.py
+ *
+ * Read more in docs/devel/block-coroutine-wrapper.rst
+ */
+#define generated_co_wrapper
+
/* block.c */
typedef struct BlockDriver BlockDriver;
typedef struct BdrvChild BdrvChild;
diff --git a/block/meson.build b/block/meson.build
index a3e56b7cd1..88ad73583a 100644
--- a/block/meson.build
+++ b/block/meson.build
@@ -107,6 +107,14 @@ module_block_h = custom_target('module_block.h',
command: [module_block_py, '@OUTPUT0@', modsrc])
block_ss.add(module_block_h)
+wrapper_py = find_program('../scripts/block-coroutine-wrapper.py')
+block_gen_c = custom_target('block-gen.c',
+ output: 'block-gen.c',
+ input: files('../include/block/block.h',
+ 'coroutines.h'),
+ command: [wrapper_py, '@OUTPUT@', '@INPUT@'])
+block_ss.add(block_gen_c)
+
block_ss.add(files('stream.c'))
softmmu_ss.add(files('qapi-sysemu.c'))
diff --git a/scripts/block-coroutine-wrapper.py b/scripts/block-coroutine-wrapper.py
new file mode 100644
index 0000000000..505e070660
--- /dev/null
+++ b/scripts/block-coroutine-wrapper.py
@@ -0,0 +1,188 @@
+#! /usr/bin/env python3
+"""Generate coroutine wrappers for block subsystem.
+
+The program parses one or several concatenated c files from stdin,
+searches for functions with the 'generated_co_wrapper' specifier
+and generates corresponding wrappers on stdout.
+
+Usage: block-coroutine-wrapper.py generated-file.c FILE.[ch]...
+
+Copyright (c) 2020 Virtuozzo International GmbH.
+
+This program is free software; you can redistribute it and/or modify
+it under the terms of the GNU General Public License as published by
+the Free Software Foundation; either version 2 of the License, or
+(at your option) any later version.
+
+This program is distributed in the hope that it will be useful,
+but WITHOUT ANY WARRANTY; without even the implied warranty of
+MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+GNU General Public License for more details.
+
+You should have received a copy of the GNU General Public License
+along with this program. If not, see <http://www.gnu.org/licenses/>.
+"""
+
+import sys
+import re
+import subprocess
+import json
+from typing import Iterator
+
+
+def prettify(code: str) -> str:
+ """Prettify code using clang-format if available"""
+
+ try:
+ style = json.dumps({
+ 'IndentWidth': 4,
+ 'BraceWrapping': {'AfterFunction': True},
+ 'BreakBeforeBraces': 'Custom',
+ 'SortIncludes': False,
+ 'MaxEmptyLinesToKeep': 2,
+ })
+ p = subprocess.run(['clang-format', f'-style={style}'], check=True,
+ encoding='utf-8', input=code,
+ stdout=subprocess.PIPE)
+ return p.stdout
+ except FileNotFoundError:
+ return code
+
+
+def gen_header():
+ copyright = re.sub('^.*Copyright', 'Copyright', __doc__, flags=re.DOTALL)
+ copyright = re.sub('^(?=.)', ' * ', copyright.strip(), flags=re.MULTILINE)
+ copyright = re.sub('^$', ' *', copyright, flags=re.MULTILINE)
+ return f"""\
+/*
+ * File is generated by scripts/block-coroutine-wrapper.py
+ *
+{copyright}
+ */
+
+#include "qemu/osdep.h"
+#include "block/coroutines.h"
+#include "block/block-gen.h"
+#include "block/block_int.h"\
+"""
+
+
+class ParamDecl:
+ param_re = re.compile(r'(?P<decl>'
+ r'(?P<type>.*[ *])'
+ r'(?P<name>[a-z][a-z0-9_]*)'
+ r')')
+
+ def __init__(self, param_decl: str) -> None:
+ m = self.param_re.match(param_decl.strip())
+ if m is None:
+ raise ValueError(f'Wrong parameter declaration: "{param_decl}"')
+ self.decl = m.group('decl')
+ self.type = m.group('type')
+ self.name = m.group('name')
+
+
+class FuncDecl:
+ def __init__(self, return_type: str, name: str, args: str) -> None:
+ self.return_type = return_type.strip()
+ self.name = name.strip()
+ self.args = [ParamDecl(arg.strip()) for arg in args.split(',')]
+
+ def gen_list(self, format: str) -> str:
+ return ', '.join(format.format_map(arg.__dict__) for arg in self.args)
+
+ def gen_block(self, format: str) -> str:
+ return '\n'.join(format.format_map(arg.__dict__) for arg in self.args)
+
+
+# Match wrappers declared with a generated_co_wrapper mark
+func_decl_re = re.compile(r'^int\s*generated_co_wrapper\s*'
+ r'(?P<wrapper_name>[a-z][a-z0-9_]*)'
+ r'\((?P<args>[^)]*)\);$', re.MULTILINE)
+
+
+def func_decl_iter(text: str) -> Iterator:
+ for m in func_decl_re.finditer(text):
+ yield FuncDecl(return_type='int',
+ name=m.group('wrapper_name'),
+ args=m.group('args'))
+
+
+def snake_to_camel(func_name: str) -> str:
+ """
+ Convert underscore names like 'some_function_name' to camel-case like
+ 'SomeFunctionName'
+ """
+ words = func_name.split('_')
+ words = [w[0].upper() + w[1:] for w in words]
+ return ''.join(words)
+
+
+def gen_wrapper(func: FuncDecl) -> str:
+ assert func.name.startswith('bdrv_')
+ assert not func.name.startswith('bdrv_co_')
+ assert func.return_type == 'int'
+ assert func.args[0].type in ['BlockDriverState *', 'BdrvChild *']
+
+ name = 'bdrv_co_' + func.name[5:]
+ bs = 'bs' if func.args[0].type == 'BlockDriverState *' else 'child->bs'
+ struct_name = snake_to_camel(name)
+
+ return f"""\
+/*
+ * Wrappers for {name}
+ */
+
+typedef struct {struct_name} {{
+ BdrvPollCo poll_state;
+{ func.gen_block(' {decl};') }
+}} {struct_name};
+
+static void coroutine_fn {name}_entry(void *opaque)
+{{
+ {struct_name} *s = opaque;
+
+ s->poll_state.ret = {name}({ func.gen_list('s->{name}') });
+ s->poll_state.in_progress = false;
+
+ aio_wait_kick();
+}}
+
+int {func.name}({ func.gen_list('{decl}') })
+{{
+ if (qemu_in_coroutine()) {{
+ return {name}({ func.gen_list('{name}') });
+ }} else {{
+ {struct_name} s = {{
+ .poll_state.bs = {bs},
+ .poll_state.in_progress = true,
+
+{ func.gen_block(' .{name} = {name},') }
+ }};
+
+ s.poll_state.co = qemu_coroutine_create({name}_entry, &s);
+
+ return bdrv_poll_co(&s.poll_state);
+ }}
+}}"""
+
+
+def gen_wrappers(input_code: str) -> str:
+ res = ''
+ for func in func_decl_iter(input_code):
+ res += '\n\n\n'
+ res += gen_wrapper(func)
+
+ return prettify(res) # prettify to wrap long lines
+
+
+if __name__ == '__main__':
+ if len(sys.argv) < 3:
+ exit(f'Usage: {sys.argv[0]} OUT_FILE.c IN_FILE.[ch]...')
+
+ with open(sys.argv[1], 'w') as f_out:
+ f_out.write(gen_header())
+ for fname in sys.argv[2:]:
+ with open(fname) as f_in:
+ f_out.write(gen_wrappers(f_in.read()))
+ f_out.write('\n')
--
2.21.3
On 9/24/20 1:54 PM, Vladimir Sementsov-Ogievskiy wrote:
> We have a very frequent pattern of creating a coroutine from a function
> with several arguments:
>
> - create a structure to pack parameters
> - create _entry function to call original function taking parameters
> from struct
> - do different magic to handle completion: set ret to NOT_DONE or
> EINPROGRESS or use separate bool field
> - fill the struct and create coroutine from _entry function with this
> struct as a parameter
> - do coroutine enter and BDRV_POLL_WHILE loop
>
> Let's reduce code duplication by generating coroutine wrappers.
>
> This patch adds scripts/block-coroutine-wrapper.py together with some
> friends, which will generate functions with declared prototypes marked
> by the 'generated_co_wrapper' specifier.
>
> The usage of new code generation is as follows:
>
> 1. define the coroutine function somewhere
>
> int coroutine_fn bdrv_co_NAME(...) {...}
>
> 2. declare in some header file
>
> int generated_co_wrapper bdrv_NAME(...);
>
> with same list of parameters (generated_co_wrapper is
> defined in "include/block/block.h").
>
> 3. Make sure the block_gen_c delaration in block/meson.build
declaration
> mentions the file with your marker function.
>
> Still, no function is now marked, this work is for the following
> commit.
>
> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
> ---
> +++ b/docs/devel/block-coroutine-wrapper.rst
> @@ -0,0 +1,54 @@
> +=======================
> +block-coroutine-wrapper
> +=======================
> +
> +A lot of functions in QEMU block layer (see ``block/*``) can only be
> +called in coroutine context. Such functions are normally marked by the
> +coroutine_fn specifier. Still, sometimes we need to call them from
> +non-coroutine context; for this we need to start a coroutine, run the
> +needed function from it and wait for coroutine finish in
to finish in a
> +BDRV_POLL_WHILE() loop. To run a coroutine we need a function with one
> +void* argument. So for each coroutine_fn function which needs a
> +non-coroutine interface, we should define a structure to pack the
> +parameters, define a separate function to unpack the parameters and
> +call the original function and finally define a new interface function
> +with same list of arguments as original one, which will pack the
> +parameters into a struct, create a coroutine, run it and wait in
> +BDRV_POLL_WHILE() loop. It's boring to create such wrappers by hand,
> +so we have a script to generate them.
> +
> +++ b/scripts/block-coroutine-wrapper.py
Reviewed-by: Eric Blake <eblake@redhat.com>
--
Eric Blake, Principal Software Engineer
Red Hat, Inc. +1-919-301-3226
Virtualization: qemu.org | libvirt.org
24.09.2020 21:54, Vladimir Sementsov-Ogievskiy wrote:
> We have a very frequent pattern of creating a coroutine from a function
> with several arguments:
>
> - create a structure to pack parameters
> - create _entry function to call original function taking parameters
> from struct
> - do different magic to handle completion: set ret to NOT_DONE or
> EINPROGRESS or use separate bool field
> - fill the struct and create coroutine from _entry function with this
> struct as a parameter
> - do coroutine enter and BDRV_POLL_WHILE loop
>
> Let's reduce code duplication by generating coroutine wrappers.
>
> This patch adds scripts/block-coroutine-wrapper.py together with some
> friends, which will generate functions with declared prototypes marked
> by the 'generated_co_wrapper' specifier.
>
> The usage of new code generation is as follows:
>
> 1. define the coroutine function somewhere
>
> int coroutine_fn bdrv_co_NAME(...) {...}
>
> 2. declare in some header file
>
> int generated_co_wrapper bdrv_NAME(...);
>
> with same list of parameters (generated_co_wrapper is
> defined in "include/block/block.h").
>
> 3. Make sure the block_gen_c delaration in block/meson.build
> mentions the file with your marker function.
>
> Still, no function is now marked, this work is for the following
> commit.
>
> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
> ---
> docs/devel/block-coroutine-wrapper.rst | 54 +++++++
> docs/devel/index.rst | 1 +
> block/block-gen.h | 49 +++++++
> include/block/block.h | 10 ++
> block/meson.build | 8 ++
> scripts/block-coroutine-wrapper.py | 188 +++++++++++++++++++++++++
> 6 files changed, 310 insertions(+)
> create mode 100644 docs/devel/block-coroutine-wrapper.rst
> create mode 100644 block/block-gen.h
> create mode 100644 scripts/block-coroutine-wrapper.py
>
> diff --git a/docs/devel/block-coroutine-wrapper.rst b/docs/devel/block-coroutine-wrapper.rst
> new file mode 100644
> index 0000000000..d09fff2cc5
> --- /dev/null
> +++ b/docs/devel/block-coroutine-wrapper.rst
> @@ -0,0 +1,54 @@
> +=======================
> +block-coroutine-wrapper
> +=======================
> +
> +A lot of functions in QEMU block layer (see ``block/*``) can only be
> +called in coroutine context. Such functions are normally marked by the
> +coroutine_fn specifier. Still, sometimes we need to call them from
> +non-coroutine context; for this we need to start a coroutine, run the
> +needed function from it and wait for coroutine finish in
> +BDRV_POLL_WHILE() loop. To run a coroutine we need a function with one
> +void* argument. So for each coroutine_fn function which needs a
> +non-coroutine interface, we should define a structure to pack the
> +parameters, define a separate function to unpack the parameters and
> +call the original function and finally define a new interface function
> +with same list of arguments as original one, which will pack the
> +parameters into a struct, create a coroutine, run it and wait in
> +BDRV_POLL_WHILE() loop. It's boring to create such wrappers by hand,
> +so we have a script to generate them.
> +
> +Usage
> +=====
> +
> +Assume we have defined the ``coroutine_fn`` function
> +``bdrv_co_foo(<some args>)`` and need a non-coroutine interface for it,
> +called ``bdrv_foo(<same args>)``. In this case the script can help. To
> +trigger the generation:
> +
> +1. You need ``bdrv_foo`` declaration somewhere (for example, in
> + ``block/coroutines.h``) with the ``generated_co_wrapper`` mark,
> + like this:
> +
> +.. code-block:: c
> +
> + int generated_co_wrapper bdrv_foo(<some args>);
> +
> +2. You need to feed this declaration to block-coroutine-wrapper script.
> + For this, add the .h (or .c) file with the declaration to the
> + ``input: files(...)`` list of ``block_gen_c`` target declaration in
> + ``block/meson.build``
> +
> +You are done. During the build, coroutine wrappers will be generated in
> +``<BUILD_DIR>/block/block-gen.c``.
> +
> +Links
> +=====
> +
> +1. The script location is ``scripts/block-coroutine-wrapper.py``.
> +
> +2. Generic place for private ``generated_co_wrapper`` declarations is
> + ``block/coroutines.h``, for public declarations:
> + ``include/block/block.h``
> +
> +3. The core API of generated coroutine wrappers is placed in
> + (not generated) ``block/block-gen.h``
> diff --git a/docs/devel/index.rst b/docs/devel/index.rst
> index 04773ce076..cb0abe1e69 100644
> --- a/docs/devel/index.rst
> +++ b/docs/devel/index.rst
> @@ -31,3 +31,4 @@ Contents:
> reset
> s390-dasd-ipl
> clocks
> + block-coroutine-wrapper
> diff --git a/block/block-gen.h b/block/block-gen.h
> new file mode 100644
> index 0000000000..f80cf4897d
> --- /dev/null
> +++ b/block/block-gen.h
> @@ -0,0 +1,49 @@
> +/*
> + * Block coroutine wrapping core, used by auto-generated block/block-gen.c
> + *
> + * Copyright (c) 2003 Fabrice Bellard
> + * Copyright (c) 2020 Virtuozzo International GmbH
> + *
> + * Permission is hereby granted, free of charge, to any person obtaining a copy
> + * of this software and associated documentation files (the "Software"), to deal
> + * in the Software without restriction, including without limitation the rights
> + * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
> + * copies of the Software, and to permit persons to whom the Software is
> + * furnished to do so, subject to the following conditions:
> + *
> + * The above copyright notice and this permission notice shall be included in
> + * all copies or substantial portions of the Software.
> + *
> + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
> + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
> + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
> + * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
> + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
> + * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
> + * THE SOFTWARE.
> + */
> +
> +#ifndef BLOCK_BLOCK_GEN_H
> +#define BLOCK_BLOCK_GEN_H
> +
> +#include "block/block_int.h"
> +
> +/* Base structure for argument packing structures */
> +typedef struct BdrvPollCo {
> + BlockDriverState *bs;
> + bool in_progress;
> + int ret;
> + Coroutine *co; /* Keep pointer here for debugging */
> +} BdrvPollCo;
> +
> +static inline int bdrv_poll_co(BdrvPollCo *s)
> +{
> + assert(!qemu_in_coroutine());
> +
> + bdrv_coroutine_enter(s->bs, s->co);
> + BDRV_POLL_WHILE(s->bs, s->in_progress);
> +
> + return s->ret;
> +}
> +
> +#endif /* BLOCK_BLOCK_GEN_H */
> diff --git a/include/block/block.h b/include/block/block.h
> index 81d591dd4c..0f0ddc51b4 100644
> --- a/include/block/block.h
> +++ b/include/block/block.h
> @@ -10,6 +10,16 @@
> #include "block/blockjob.h"
> #include "qemu/hbitmap.h"
>
> +/*
> + * generated_co_wrapper
> + *
> + * Function specifier, which does nothing but mark functions to be
> + * generated by scripts/block-coroutine-wrapper.py
> + *
> + * Read more in docs/devel/block-coroutine-wrapper.rst
> + */
> +#define generated_co_wrapper
> +
> /* block.c */
> typedef struct BlockDriver BlockDriver;
> typedef struct BdrvChild BdrvChild;
> diff --git a/block/meson.build b/block/meson.build
> index a3e56b7cd1..88ad73583a 100644
> --- a/block/meson.build
> +++ b/block/meson.build
> @@ -107,6 +107,14 @@ module_block_h = custom_target('module_block.h',
> command: [module_block_py, '@OUTPUT0@', modsrc])
> block_ss.add(module_block_h)
>
> +wrapper_py = find_program('../scripts/block-coroutine-wrapper.py')
> +block_gen_c = custom_target('block-gen.c',
> + output: 'block-gen.c',
> + input: files('../include/block/block.h',
> + 'coroutines.h'),
> + command: [wrapper_py, '@OUTPUT@', '@INPUT@'])
> +block_ss.add(block_gen_c)
> +
> block_ss.add(files('stream.c'))
>
> softmmu_ss.add(files('qapi-sysemu.c'))
> diff --git a/scripts/block-coroutine-wrapper.py b/scripts/block-coroutine-wrapper.py
> new file mode 100644
> index 0000000000..505e070660
> --- /dev/null
> +++ b/scripts/block-coroutine-wrapper.py
> @@ -0,0 +1,188 @@
> +#! /usr/bin/env python3
> +"""Generate coroutine wrappers for block subsystem.
> +
> +The program parses one or several concatenated c files from stdin,
> +searches for functions with the 'generated_co_wrapper' specifier
> +and generates corresponding wrappers on stdout.
> +
> +Usage: block-coroutine-wrapper.py generated-file.c FILE.[ch]...
> +
> +Copyright (c) 2020 Virtuozzo International GmbH.
> +
> +This program is free software; you can redistribute it and/or modify
> +it under the terms of the GNU General Public License as published by
> +the Free Software Foundation; either version 2 of the License, or
> +(at your option) any later version.
> +
> +This program is distributed in the hope that it will be useful,
> +but WITHOUT ANY WARRANTY; without even the implied warranty of
> +MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> +GNU General Public License for more details.
> +
> +You should have received a copy of the GNU General Public License
> +along with this program. If not, see <http://www.gnu.org/licenses/>.
> +"""
> +
> +import sys
> +import re
> +import subprocess
> +import json
> +from typing import Iterator
> +
> +
> +def prettify(code: str) -> str:
> + """Prettify code using clang-format if available"""
> +
> + try:
> + style = json.dumps({
> + 'IndentWidth': 4,
> + 'BraceWrapping': {'AfterFunction': True},
> + 'BreakBeforeBraces': 'Custom',
> + 'SortIncludes': False,
> + 'MaxEmptyLinesToKeep': 2,
> + })
> + p = subprocess.run(['clang-format', f'-style={style}'], check=True,
> + encoding='utf-8', input=code,
> + stdout=subprocess.PIPE)
> + return p.stdout
> + except FileNotFoundError:
> + return code
> +
> +
> +def gen_header():
> + copyright = re.sub('^.*Copyright', 'Copyright', __doc__, flags=re.DOTALL)
> + copyright = re.sub('^(?=.)', ' * ', copyright.strip(), flags=re.MULTILINE)
> + copyright = re.sub('^$', ' *', copyright, flags=re.MULTILINE)
> + return f"""\
> +/*
> + * File is generated by scripts/block-coroutine-wrapper.py
> + *
> +{copyright}
> + */
> +
> +#include "qemu/osdep.h"
> +#include "block/coroutines.h"
> +#include "block/block-gen.h"
> +#include "block/block_int.h"\
> +"""
> +
> +
> +class ParamDecl:
> + param_re = re.compile(r'(?P<decl>'
> + r'(?P<type>.*[ *])'
> + r'(?P<name>[a-z][a-z0-9_]*)'
> + r')')
> +
> + def __init__(self, param_decl: str) -> None:
> + m = self.param_re.match(param_decl.strip())
> + if m is None:
> + raise ValueError(f'Wrong parameter declaration: "{param_decl}"')
> + self.decl = m.group('decl')
> + self.type = m.group('type')
> + self.name = m.group('name')
> +
> +
> +class FuncDecl:
> + def __init__(self, return_type: str, name: str, args: str) -> None:
> + self.return_type = return_type.strip()
> + self.name = name.strip()
> + self.args = [ParamDecl(arg.strip()) for arg in args.split(',')]
> +
> + def gen_list(self, format: str) -> str:
> + return ', '.join(format.format_map(arg.__dict__) for arg in self.args)
> +
> + def gen_block(self, format: str) -> str:
> + return '\n'.join(format.format_map(arg.__dict__) for arg in self.args)
> +
> +
> +# Match wrappers declared with a generated_co_wrapper mark
> +func_decl_re = re.compile(r'^int\s*generated_co_wrapper\s*'
> + r'(?P<wrapper_name>[a-z][a-z0-9_]*)'
> + r'\((?P<args>[^)]*)\);$', re.MULTILINE)
> +
> +
> +def func_decl_iter(text: str) -> Iterator:
> + for m in func_decl_re.finditer(text):
> + yield FuncDecl(return_type='int',
> + name=m.group('wrapper_name'),
> + args=m.group('args'))
> +
> +
> +def snake_to_camel(func_name: str) -> str:
> + """
> + Convert underscore names like 'some_function_name' to camel-case like
> + 'SomeFunctionName'
> + """
> + words = func_name.split('_')
> + words = [w[0].upper() + w[1:] for w in words]
> + return ''.join(words)
> +
> +
> +def gen_wrapper(func: FuncDecl) -> str:
> + assert func.name.startswith('bdrv_')
> + assert not func.name.startswith('bdrv_co_')
> + assert func.return_type == 'int'
> + assert func.args[0].type in ['BlockDriverState *', 'BdrvChild *']
> +
> + name = 'bdrv_co_' + func.name[5:]
> + bs = 'bs' if func.args[0].type == 'BlockDriverState *' else 'child->bs'
> + struct_name = snake_to_camel(name)
> +
> + return f"""\
> +/*
> + * Wrappers for {name}
> + */
> +
> +typedef struct {struct_name} {{
> + BdrvPollCo poll_state;
> +{ func.gen_block(' {decl};') }
> +}} {struct_name};
> +
> +static void coroutine_fn {name}_entry(void *opaque)
> +{{
> + {struct_name} *s = opaque;
> +
> + s->poll_state.ret = {name}({ func.gen_list('s->{name}') });
> + s->poll_state.in_progress = false;
> +
> + aio_wait_kick();
> +}}
> +
> +int {func.name}({ func.gen_list('{decl}') })
> +{{
> + if (qemu_in_coroutine()) {{
> + return {name}({ func.gen_list('{name}') });
> + }} else {{
> + {struct_name} s = {{
> + .poll_state.bs = {bs},
> + .poll_state.in_progress = true,
> +
> +{ func.gen_block(' .{name} = {name},') }
> + }};
> +
> + s.poll_state.co = qemu_coroutine_create({name}_entry, &s);
> +
> + return bdrv_poll_co(&s.poll_state);
> + }}
> +}}"""
> +
> +
> +def gen_wrappers(input_code: str) -> str:
> + res = ''
> + for func in func_decl_iter(input_code):
> + res += '\n\n\n'
> + res += gen_wrapper(func)
> +
> + return prettify(res) # prettify to wrap long lines
> +
> +
> +if __name__ == '__main__':
> + if len(sys.argv) < 3:
> + exit(f'Usage: {sys.argv[0]} OUT_FILE.c IN_FILE.[ch]...')
> +
> + with open(sys.argv[1], 'w') as f_out:
> + f_out.write(gen_header())
> + for fname in sys.argv[2:]:
> + with open(fname) as f_in:
> + f_out.write(gen_wrappers(f_in.read()))
> + f_out.write('\n')
>
probably we want encoding='utf-8' for both open calls.
--
Best regards,
Vladimir
© 2016 - 2026 Red Hat, Inc.