Hi

While discussing running more tests for CoreSight, we thought it might
be a good idea to upstream some EndToEnd tests for CoreSight decoding in
order to lock down the behaviour. I am sending this as RFC to get some
feedback from the community first.

The test relies on pre-geneated perf.data files that are downloaded
during the test. I'm not sure it's a good idea to commit those files to
the Linux repository, so they would have to live in an external source
and be downloaded during the test.

For this RFC, the files are stored in a Github repository [1]. As an
idea, I think we could store them in a new repo in the ARM-software
namespace. Any hosting suggestions are very welcome.

Thanks,
German

[1] https://github.com/ARM-software/data/tree/984cde8fb0bb22591e284826a80b338bb79c3655/perf/coresight

German Gomez (1):
  perf test cs-etm: Add end-to-end tests for CoreSight decoding

 tools/perf/tests/shell/lib/arm_auxtrace.sh    | 21 +++++++
 .../tests/shell/test_arm_coresight_decoder.sh | 57 +++++++++++++++++++
 2 files changed, 78 insertions(+)
 create mode 100644 tools/perf/tests/shell/lib/arm_auxtrace.sh
 create mode 100755 tools/perf/tests/shell/test_arm_coresight_decoder.sh

-- 
2.25.1

Em Mon, May 23, 2022 at 03:49:51PM +0100, German Gomez escreveu:
> Hi
> 
> While discussing running more tests for CoreSight, we thought it might
> be a good idea to upstream some EndToEnd tests for CoreSight decoding in
> order to lock down the behaviour. I am sending this as RFC to get some
> feedback from the community first.
> 
> The test relies on pre-geneated perf.data files that are downloaded
> during the test. I'm not sure it's a good idea to commit those files to
> the Linux repository, so they would have to live in an external source
> and be downloaded during the test.

That is ok, but please cache it locally, so that from the second 'perf
test' run onwards one doesn´t have to incur in the download delay and
also be able to run the tests while not connected.

- Arnaldo
 
> For this RFC, the files are stored in a Github repository [1]. As an
> idea, I think we could store them in a new repo in the ARM-software
> namespace. Any hosting suggestions are very welcome.
> 
> Thanks,
> German
> 
> [1] https://github.com/ARM-software/data/tree/984cde8fb0bb22591e284826a80b338bb79c3655/perf/coresight
> 
> German Gomez (1):
>   perf test cs-etm: Add end-to-end tests for CoreSight decoding
> 
>  tools/perf/tests/shell/lib/arm_auxtrace.sh    | 21 +++++++
>  .../tests/shell/test_arm_coresight_decoder.sh | 57 +++++++++++++++++++
>  2 files changed, 78 insertions(+)
>  create mode 100644 tools/perf/tests/shell/lib/arm_auxtrace.sh
>  create mode 100755 tools/perf/tests/shell/test_arm_coresight_decoder.sh
> 
> -- 
> 2.25.1

-- 

- Arnaldo

On Mon, May 23, 2022 at 1:05 PM Arnaldo Carvalho de Melo
<acme@kernel.org> wrote:
>
> Em Mon, May 23, 2022 at 03:49:51PM +0100, German Gomez escreveu:
> > Hi
> >
> > While discussing running more tests for CoreSight, we thought it might
> > be a good idea to upstream some EndToEnd tests for CoreSight decoding in
> > order to lock down the behaviour. I am sending this as RFC to get some
> > feedback from the community first.
> >
> > The test relies on pre-geneated perf.data files that are downloaded
> > during the test. I'm not sure it's a good idea to commit those files to
> > the Linux repository, so they would have to live in an external source
> > and be downloaded during the test.
>
> That is ok, but please cache it locally, so that from the second 'perf
> test' run onwards one doesn´t have to incur in the download delay and
> also be able to run the tests while not connected.

I have some continuous tests running that don't have internet access.
Could we have an environment variable to give a path to an already
downloaded version? I may be able to fake having a cached downloaded
version. Where would such a download live?

Thanks,
Ian

> - Arnaldo
>
> > For this RFC, the files are stored in a Github repository [1]. As an
> > idea, I think we could store them in a new repo in the ARM-software
> > namespace. Any hosting suggestions are very welcome.
> >
> > Thanks,
> > German
> >
> > [1] https://github.com/ARM-software/data/tree/984cde8fb0bb22591e284826a80b338bb79c3655/perf/coresight
> >
> > German Gomez (1):
> >   perf test cs-etm: Add end-to-end tests for CoreSight decoding
> >
> >  tools/perf/tests/shell/lib/arm_auxtrace.sh    | 21 +++++++
> >  .../tests/shell/test_arm_coresight_decoder.sh | 57 +++++++++++++++++++
> >  2 files changed, 78 insertions(+)
> >  create mode 100644 tools/perf/tests/shell/lib/arm_auxtrace.sh
> >  create mode 100755 tools/perf/tests/shell/test_arm_coresight_decoder.sh
> >
> > --
> > 2.25.1
>
> --
>
> - Arnaldo

Hi Ian, Arnaldo, thanks a lot for your feedback

On 23/05/2022 22:29, Ian Rogers wrote:
> On Mon, May 23, 2022 at 1:05 PM Arnaldo Carvalho de Melo
> <acme@kernel.org> wrote:
>> Em Mon, May 23, 2022 at 03:49:51PM +0100, German Gomez escreveu:
>>> Hi
>>>
>>> While discussing running more tests for CoreSight, we thought it might
>>> be a good idea to upstream some EndToEnd tests for CoreSight decoding in
>>> order to lock down the behaviour. I am sending this as RFC to get some
>>> feedback from the community first.
>>>
>>> The test relies on pre-geneated perf.data files that are downloaded
>>> during the test. I'm not sure it's a good idea to commit those files to
>>> the Linux repository, so they would have to live in an external source
>>> and be downloaded during the test.
>> That is ok, but please cache it locally, so that from the second 'perf
>> test' run onwards one doesn´t have to incur in the download delay and
>> also be able to run the tests while not connected.

Agreed, I will keep this in mind.

> I have some continuous tests running that don't have internet access.
> Could we have an environment variable to give a path to an already
> downloaded version? I may be able to fake having a cached downloaded
> version. Where would such a download live?

Yes sure, download location could be (in order of priority) $PERF_TEST_AUXTRACE_DECODER > $XDG_CACHE_HOME > ~/.cache. I can add a mechanism to override the download step if needed.

Also I think it would make sense for it to be generic/extensible for SPE and PT as well.

Imho best location for Arm files at the moment is github/ARM-software, since we already kind of use it for perf work (the pmu json files, for example). We also have gitlab.arm.com but I'm not very familiar with that one, or if it can be used at all.

Thanks,
German

>
> Thanks,
> Ian
>
>> - Arnaldo
>>
>>> For this RFC, the files are stored in a Github repository [1]. As an
>>> idea, I think we could store them in a new repo in the ARM-software
>>> namespace. Any hosting suggestions are very welcome.
>>>
>>> Thanks,
>>> German
>>>
>>> [1] https://github.com/ARM-software/data/tree/984cde8fb0bb22591e284826a80b338bb79c3655/perf/coresight
>>>
>>> German Gomez (1):
>>>   perf test cs-etm: Add end-to-end tests for CoreSight decoding
>>>
>>>  tools/perf/tests/shell/lib/arm_auxtrace.sh    | 21 +++++++
>>>  .../tests/shell/test_arm_coresight_decoder.sh | 57 +++++++++++++++++++
>>>  2 files changed, 78 insertions(+)
>>>  create mode 100644 tools/perf/tests/shell/lib/arm_auxtrace.sh
>>>  create mode 100755 tools/perf/tests/shell/test_arm_coresight_decoder.sh
>>>
>>> --
>>> 2.25.1
>> --
>>
>> - Arnaldo