[Qemu-devel] [RFC 0/6] initial plugin support

Emilio G. Cota posted 6 patches 6 years, 6 months ago
Failed in applying to current master (apply log)
Makefile                  |   7 +-
Makefile.objs             |   1 +
accel/tcg/cpu-exec.c      |   6 +-
accel/tcg/translate-all.c |   9 +-
configure                 |  52 ++++
include/exec/exec-all.h   |   2 +
include/exec/tb-hash-xx.h |  26 +-
include/exec/tb-hash.h    |   6 +-
include/exec/tb-lookup.h  |   1 +
include/qemu/plugin-api.h | 105 +++++++
include/qemu/plugin.h     |  74 +++++
include/qom/cpu.h         |   4 +
linux-user/main.c         |  18 ++
plugin.c                  | 519 ++++++++++++++++++++++++++++++++
qemu-options.hx           |  17 ++
qemu-plugins.symbols      |   6 +
qom/cpu.c                 |   3 +
tcg/tcg-op.c              |  10 +-
tcg/tcg.h                 |   2 +-
tests/qht-bench.c         |   2 +-
vl.c                      |  10 +
21 files changed, 857 insertions(+), 23 deletions(-)
[Qemu-devel] [RFC 0/6] initial plugin support
Posted by Emilio G. Cota 6 years, 6 months ago
Related threads:
  [PATCH 00/13] instrument: Add basic event instrumentation
  Date: Mon, 24 Jul 2017 20:02:24 +0300
  https://lists.gnu.org/archive/html/qemu-devel/2017-07/msg07419.html
and
  [PATCH v4 00/20] instrument: Add basic event instrumentation
  Date:	Wed, 6 Sep 2017 20:22:41 +0300
  https://lists.gnu.org/archive/html/qemu-devel/2017-07/msg07419.html

This set does something similar to the instrumentation patches by Lluis,
but with a different implementation (and for now less events).

My focus has been on working on the skeleton of a (pseudo) stable API,
as Stefan requested. Of course more events would have to be added, but
before spending more time on this I'd like to get some feedback on the
core of the design. Patch 2 has all the details.

Note: yes, patch 1 is not used in the series, but this is an RFC. It's there
because it will be needed to get the tb->plugin_mask when deciding whether
to generate a mem_cb helper when generating loads/stores from TCG.

This set applies on top of:
  https://github.com/cota/qemu/tree/tcg-generic-15%2Bmulti-tcg-v4-parallel

The tree can be fetched from:
  https://github.com/cota/qemu/tree/plugins

Thanks,

		Emilio
---
 Makefile                  |   7 +-
 Makefile.objs             |   1 +
 accel/tcg/cpu-exec.c      |   6 +-
 accel/tcg/translate-all.c |   9 +-
 configure                 |  52 ++++
 include/exec/exec-all.h   |   2 +
 include/exec/tb-hash-xx.h |  26 +-
 include/exec/tb-hash.h    |   6 +-
 include/exec/tb-lookup.h  |   1 +
 include/qemu/plugin-api.h | 105 +++++++
 include/qemu/plugin.h     |  74 +++++
 include/qom/cpu.h         |   4 +
 linux-user/main.c         |  18 ++
 plugin.c                  | 519 ++++++++++++++++++++++++++++++++
 qemu-options.hx           |  17 ++
 qemu-plugins.symbols      |   6 +
 qom/cpu.c                 |   3 +
 tcg/tcg-op.c              |  10 +-
 tcg/tcg.h                 |   2 +-
 tests/qht-bench.c         |   2 +-
 vl.c                      |  10 +
 21 files changed, 857 insertions(+), 23 deletions(-)

Re: [Qemu-devel] [RFC 0/6] initial plugin support
Posted by Peter Maydell 6 years, 6 months ago
On 6 September 2017 at 21:28, Emilio G. Cota <cota@braap.org> wrote:
> Related threads:
>   [PATCH 00/13] instrument: Add basic event instrumentation
>   Date: Mon, 24 Jul 2017 20:02:24 +0300
>   https://lists.gnu.org/archive/html/qemu-devel/2017-07/msg07419.html
> and
>   [PATCH v4 00/20] instrument: Add basic event instrumentation
>   Date: Wed, 6 Sep 2017 20:22:41 +0300
>   https://lists.gnu.org/archive/html/qemu-devel/2017-07/msg07419.html
>
> This set does something similar to the instrumentation patches by Lluis,
> but with a different implementation (and for now less events).
>
> My focus has been on working on the skeleton of a (pseudo) stable API,
> as Stefan requested. Of course more events would have to be added, but
> before spending more time on this I'd like to get some feedback on the
> core of the design. Patch 2 has all the details.
>
> Note: yes, patch 1 is not used in the series, but this is an RFC. It's there
> because it will be needed to get the tb->plugin_mask when deciding whether
> to generate a mem_cb helper when generating loads/stores from TCG.
>
> This set applies on top of:
>   https://github.com/cota/qemu/tree/tcg-generic-15%2Bmulti-tcg-v4-parallel
>
> The tree can be fetched from:
>   https://github.com/cota/qemu/tree/plugins

Hi -- do you have documentation of what the plugin-facing
API here is? What I'd like to do for the initial evaluation of
this series and Luis's is just to look at the plugin API
and the command line/etc interface for users to provide the
API, because I think that's the interesting part.

thanks
-- PMM

Re: [Qemu-devel] [RFC 0/6] initial plugin support
Posted by Thomas Huth 6 years, 6 months ago
On 06.09.2017 22:28, Emilio G. Cota wrote:
> Related threads:
>   [PATCH 00/13] instrument: Add basic event instrumentation
>   Date: Mon, 24 Jul 2017 20:02:24 +0300
>   https://lists.gnu.org/archive/html/qemu-devel/2017-07/msg07419.html
> and
>   [PATCH v4 00/20] instrument: Add basic event instrumentation
>   Date:	Wed, 6 Sep 2017 20:22:41 +0300
>   https://lists.gnu.org/archive/html/qemu-devel/2017-07/msg07419.html
> 
> This set does something similar to the instrumentation patches by Lluis,
> but with a different implementation (and for now less events).
> 
> My focus has been on working on the skeleton of a (pseudo) stable API,
> as Stefan requested. Of course more events would have to be added, but
> before spending more time on this I'd like to get some feedback on the
> core of the design. Patch 2 has all the details.

Sorry for my ignorance, but if you send a patch series like this, could
you please elaborate a little bit more on the topic what this all is
about? In this cover letter, you basically give only some pointers about
other patch series and point the reader to patch 2, but also patch 2
does not really have a proper *description* of what this is really all
about. Sure, it's about plugins, but what kind of plugins? Audio? Video?
CPU? Everything? If you send RFC, you should properly describe your
vision first, and maybe give some examples, before you jump into the
details.

 Thanks,
  Thomas

Re: [Qemu-devel] [RFC 0/6] initial plugin support
Posted by Stefan Hajnoczi 6 years, 6 months ago
On Tue, Sep 26, 2017 at 06:22:31AM +0200, Thomas Huth wrote:
> On 06.09.2017 22:28, Emilio G. Cota wrote:
> > Related threads:
> >   [PATCH 00/13] instrument: Add basic event instrumentation
> >   Date: Mon, 24 Jul 2017 20:02:24 +0300
> >   https://lists.gnu.org/archive/html/qemu-devel/2017-07/msg07419.html
> > and
> >   [PATCH v4 00/20] instrument: Add basic event instrumentation
> >   Date:	Wed, 6 Sep 2017 20:22:41 +0300
> >   https://lists.gnu.org/archive/html/qemu-devel/2017-07/msg07419.html
> > 
> > This set does something similar to the instrumentation patches by Lluis,
> > but with a different implementation (and for now less events).
> > 
> > My focus has been on working on the skeleton of a (pseudo) stable API,
> > as Stefan requested. Of course more events would have to be added, but
> > before spending more time on this I'd like to get some feedback on the
> > core of the design. Patch 2 has all the details.
> 
> Sorry for my ignorance, but if you send a patch series like this, could
> you please elaborate a little bit more on the topic what this all is
> about? In this cover letter, you basically give only some pointers about
> other patch series and point the reader to patch 2, but also patch 2
> does not really have a proper *description* of what this is really all
> about. Sure, it's about plugins, but what kind of plugins? Audio? Video?
> CPU? Everything? If you send RFC, you should properly describe your
> vision first, and maybe give some examples, before you jump into the
> details.

Good point.

This series is about introducing a stable API/ABI and loadable
instrumentation plugin interface.  The goal is to offer dynamic binary
instrumentation frameworks like Intel's Pin and DynamoRIO offer.

In other words, performance analysis tasks (especially for TCG) can be
implemented in a shared library that is loaded by QEMU when running a
benchmark.

Stefan

Re: [Qemu-devel] [RFC 0/6] initial plugin support
Posted by Stefan Hajnoczi 6 years, 6 months ago
On Wed, Sep 06, 2017 at 04:28:42PM -0400, Emilio G. Cota wrote:
> Related threads:
>   [PATCH 00/13] instrument: Add basic event instrumentation
>   Date: Mon, 24 Jul 2017 20:02:24 +0300
>   https://lists.gnu.org/archive/html/qemu-devel/2017-07/msg07419.html
> and
>   [PATCH v4 00/20] instrument: Add basic event instrumentation
>   Date:	Wed, 6 Sep 2017 20:22:41 +0300
>   https://lists.gnu.org/archive/html/qemu-devel/2017-07/msg07419.html
> 
> This set does something similar to the instrumentation patches by Lluis,
> but with a different implementation (and for now less events).
> 
> My focus has been on working on the skeleton of a (pseudo) stable API,
> as Stefan requested. Of course more events would have to be added, but
> before spending more time on this I'd like to get some feedback on the
> core of the design. Patch 2 has all the details.
> 
> Note: yes, patch 1 is not used in the series, but this is an RFC. It's there
> because it will be needed to get the tb->plugin_mask when deciding whether
> to generate a mem_cb helper when generating loads/stores from TCG.
> 
> This set applies on top of:
>   https://github.com/cota/qemu/tree/tcg-generic-15%2Bmulti-tcg-v4-parallel
> 
> The tree can be fetched from:
>   https://github.com/cota/qemu/tree/plugins

Hi Emilio,
Thanks for sending this!  Could you and Lluís collaborate on a single
instrumentation plugin API?

Let's have a discussion with Lluís, Peter, myself, and potentially
others then we can use the QEMU Developer Conference Call that is
scheduled every 2 weeks.

The next call time slot is Tue 26 Sept at 13:00 UTC:
https://www.timeanddate.com/worldclock/fixedtime.html?iso=20170926T140000&p1=136

Stefan

Re: [Qemu-devel] [RFC 0/6] initial plugin support
Posted by Peter Maydell 6 years, 6 months ago
On 18 September 2017 at 15:59, Stefan Hajnoczi <stefanha@gmail.com> wrote:
> Thanks for sending this!  Could you and Lluís collaborate on a single
> instrumentation plugin API?
>
> Let's have a discussion with Lluís, Peter, myself, and potentially
> others then we can use the QEMU Developer Conference Call that is
> scheduled every 2 weeks.
>
> The next call time slot is Tue 26 Sept at 13:00 UTC:
> https://www.timeanddate.com/worldclock/fixedtime.html?iso=20170926T140000&p1=136

I can't make that, I'm afraid, it's Linaro Connect that week.

From my point of view, it would be helpful to start with
just designing the API that we're trying to implement.
I care a lot that we get that API correct, I don't care much
at all about the implementation...

thanks
-- PMM

Re: [Qemu-devel] [RFC 0/6] initial plugin support
Posted by Stefan Hajnoczi 6 years, 6 months ago
On Mon, Sep 18, 2017 at 04:05:31PM +0100, Peter Maydell wrote:
> On 18 September 2017 at 15:59, Stefan Hajnoczi <stefanha@gmail.com> wrote:
> > Thanks for sending this!  Could you and Lluís collaborate on a single
> > instrumentation plugin API?
> >
> > Let's have a discussion with Lluís, Peter, myself, and potentially
> > others then we can use the QEMU Developer Conference Call that is
> > scheduled every 2 weeks.
> >
> > The next call time slot is Tue 26 Sept at 13:00 UTC:
> > https://www.timeanddate.com/worldclock/fixedtime.html?iso=20170926T140000&p1=136
> 
> I can't make that, I'm afraid, it's Linaro Connect that week.
> 
> From my point of view, it would be helpful to start with
> just designing the API that we're trying to implement.
> I care a lot that we get that API correct, I don't care much
> at all about the implementation...

That sounds good to me.

An RFC patch with just the header files would be enough for mailing list
discussion.  No implementation is needed yet.

Stefan

Re: [Qemu-devel] [RFC 0/6] initial plugin support
Posted by Lluís Vilanova 6 years, 6 months ago
Stefan Hajnoczi writes:

> On Mon, Sep 18, 2017 at 04:05:31PM +0100, Peter Maydell wrote:
>> On 18 September 2017 at 15:59, Stefan Hajnoczi <stefanha@gmail.com> wrote:
>> > Thanks for sending this!  Could you and Lluís collaborate on a single
>> > instrumentation plugin API?
>> >
>> > Let's have a discussion with Lluís, Peter, myself, and potentially
>> > others then we can use the QEMU Developer Conference Call that is
>> > scheduled every 2 weeks.
>> >
>> > The next call time slot is Tue 26 Sept at 13:00 UTC:
>> > https://www.timeanddate.com/worldclock/fixedtime.html?iso=20170926T140000&p1=136
>> 
>> I can't make that, I'm afraid, it's Linaro Connect that week.
>> 
>> From my point of view, it would be helpful to start with
>> just designing the API that we're trying to implement.
>> I care a lot that we get that API correct, I don't care much
>> at all about the implementation...

> That sounds good to me.

> An RFC patch with just the header files would be enough for mailing list
> discussion.  No implementation is needed yet.

I can send a stripped down version of this series only with the docs and public
headers.

But first I would like to know how to handle the translate/execute
differentiation first (if it is indeed needed).


Cheers,
  Lluis