1. Patchew
  2. QEMU
  3. View series

[Qemu-devel] [PATCH] tests/tcg: add simple record/replay smoke test for aarch64

Alex Bennée posted 1 patch 4 years, 7 months ago
Test FreeBSD passed
Test docker-mingw@fedora passed
Test asan failed
Test docker-clang@ubuntu passed
Test checkpatch passed
Test docker-quick@centos7 passed
Patches applied successfully (tree, apply log)
git fetch https://github.com/patchew-project/qemu tags/patchew/20190918153335.20797-1-alex.bennee@linaro.org
tests/tcg/aarch64/Makefile.softmmu-target | 21 +++++++++++++++++++++
1 file changed, 21 insertions(+)
[Qemu-devel] [PATCH] tests/tcg: add simple record/replay smoke test for aarch64
Posted by Alex Bennée 4 years, 7 months ago 
This adds two new tests that re-use the memory test to check basic
record replay functionality is still working. We have to define our
own runners rather than using the default pattern as we want to change
the test name but re-use the memory binary.

We declare the test binaries as PHONY as they don't rely exist.

[AJB: A better test would output some sort of timer value or other
otherwise variable value so we could compare the record and replay
outputs and ensure they match]

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Cc: Pavel Dovgalyuk <dovgaluk@ispras.ru>
---
 tests/tcg/aarch64/Makefile.softmmu-target | 21 +++++++++++++++++++++
 1 file changed, 21 insertions(+)

diff --git a/tests/tcg/aarch64/Makefile.softmmu-target b/tests/tcg/aarch64/Makefile.softmmu-target
index 4c4aaf61dd3..b4b39579634 100644
--- a/tests/tcg/aarch64/Makefile.softmmu-target
+++ b/tests/tcg/aarch64/Makefile.softmmu-target
@@ -32,3 +32,24 @@ memory: CFLAGS+=-DCHECK_UNALIGNED=1
 
 # Running
 QEMU_OPTS+=-M virt -cpu max -display none -semihosting-config enable=on,target=native,chardev=output -kernel
+
+# Simple Record/Replay Test
+.PHONY: memory-record
+run-memory-record: memory-record memory
+	$(call run-test, $<, \
+	  $(QEMU) -monitor none -display none \
+		  -chardev file$(COMMA)path=$<.out$(COMMA)id=output \
+		  -icount shift=5$(COMMA)rr=record$(COMMA)rrfile=record.bin \
+	   	  $(QEMU_OPTS) memory, \
+	  "$< on $(TARGET_NAME)")
+
+.PHONY: memory-replay
+run-memory-replay: memory-replay run-memory-record
+	$(call run-test, $<, \
+	  $(QEMU) -monitor none -display none \
+		  -chardev file$(COMMA)path=$<.out$(COMMA)id=output \
+		  -icount shift=5$(COMMA)rr=replay$(COMMA)rrfile=record.bin \
+	   	  $(QEMU_OPTS) memory, \
+	  "$< on $(TARGET_NAME)")
+
+TESTS+=memory-record memory-replay
-- 
2.20.1
Re: [Qemu-devel] [PATCH] tests/tcg: add simple record/replay smoke test for aarch64
Posted by Pavel Dovgalyuk 4 years, 7 months ago 
Thanks!

This seem to correctly run record and replay command lines.
When I break the replay correctness, then the test reports a timeout error.

However, we need some kind of a manual for tcg testing. I had to dig through makefile and configure
scripts to undestand that testing needs the cross compilers to be installed.
Then I installed a random cross compiler and everything worked normally.

Pavel Dovgalyuk

> -----Original Message-----
> From: Alex Bennée [mailto:alex.bennee@linaro.org]
> Sent: Wednesday, September 18, 2019 6:34 PM
> To: qemu-devel@nongnu.org
> Cc: Alex Bennée; Pavel Dovgalyuk; Peter Maydell; open list:ARM TCG CPUs
> Subject: [PATCH] tests/tcg: add simple record/replay smoke test for aarch64
> 
> This adds two new tests that re-use the memory test to check basic
> record replay functionality is still working. We have to define our
> own runners rather than using the default pattern as we want to change
> the test name but re-use the memory binary.
> 
> We declare the test binaries as PHONY as they don't rely exist.
> 
> [AJB: A better test would output some sort of timer value or other
> otherwise variable value so we could compare the record and replay
> outputs and ensure they match]
> 
> Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
> Cc: Pavel Dovgalyuk <dovgaluk@ispras.ru>
> ---
>  tests/tcg/aarch64/Makefile.softmmu-target | 21 +++++++++++++++++++++
>  1 file changed, 21 insertions(+)
> 
> diff --git a/tests/tcg/aarch64/Makefile.softmmu-target b/tests/tcg/aarch64/Makefile.softmmu-
> target
> index 4c4aaf61dd3..b4b39579634 100644
> --- a/tests/tcg/aarch64/Makefile.softmmu-target
> +++ b/tests/tcg/aarch64/Makefile.softmmu-target
> @@ -32,3 +32,24 @@ memory: CFLAGS+=-DCHECK_UNALIGNED=1
> 
>  # Running
>  QEMU_OPTS+=-M virt -cpu max -display none -semihosting-config
> enable=on,target=native,chardev=output -kernel
> +
> +# Simple Record/Replay Test
> +.PHONY: memory-record
> +run-memory-record: memory-record memory
> +	$(call run-test, $<, \
> +	  $(QEMU) -monitor none -display none \
> +		  -chardev file$(COMMA)path=$<.out$(COMMA)id=output \
> +		  -icount shift=5$(COMMA)rr=record$(COMMA)rrfile=record.bin \
> +	   	  $(QEMU_OPTS) memory, \
> +	  "$< on $(TARGET_NAME)")
> +
> +.PHONY: memory-replay
> +run-memory-replay: memory-replay run-memory-record
> +	$(call run-test, $<, \
> +	  $(QEMU) -monitor none -display none \
> +		  -chardev file$(COMMA)path=$<.out$(COMMA)id=output \
> +		  -icount shift=5$(COMMA)rr=replay$(COMMA)rrfile=record.bin \
> +	   	  $(QEMU_OPTS) memory, \
> +	  "$< on $(TARGET_NAME)")
> +
> +TESTS+=memory-record memory-replay
> --
> 2.20.1
Re: [Qemu-devel] [PATCH] tests/tcg: add simple record/replay smoke test for aarch64
Posted by Alex Bennée 4 years, 7 months ago 
Pavel Dovgalyuk <dovgaluk@ispras.ru> writes:

> Thanks!
>
> This seem to correctly run record and replay command lines.
> When I break the replay correctness, then the test reports a timeout error.
>
> However, we need some kind of a manual for tcg testing. I had to dig through makefile and configure
> scripts to undestand that testing needs the cross compilers to be installed.
> Then I installed a random cross compiler and everything worked
> normally.

I shall add a section to the testing.rst documentation.

>
> Pavel Dovgalyuk
>
>> -----Original Message-----
>> From: Alex Bennée [mailto:alex.bennee@linaro.org]
>> Sent: Wednesday, September 18, 2019 6:34 PM
>> To: qemu-devel@nongnu.org
>> Cc: Alex Bennée; Pavel Dovgalyuk; Peter Maydell; open list:ARM TCG CPUs
>> Subject: [PATCH] tests/tcg: add simple record/replay smoke test for aarch64
>>
>> This adds two new tests that re-use the memory test to check basic
>> record replay functionality is still working. We have to define our
>> own runners rather than using the default pattern as we want to change
>> the test name but re-use the memory binary.
>>
>> We declare the test binaries as PHONY as they don't rely exist.
>>
>> [AJB: A better test would output some sort of timer value or other
>> otherwise variable value so we could compare the record and replay
>> outputs and ensure they match]
>>
>> Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
>> Cc: Pavel Dovgalyuk <dovgaluk@ispras.ru>
>> ---
>>  tests/tcg/aarch64/Makefile.softmmu-target | 21 +++++++++++++++++++++
>>  1 file changed, 21 insertions(+)
>>
>> diff --git a/tests/tcg/aarch64/Makefile.softmmu-target b/tests/tcg/aarch64/Makefile.softmmu-
>> target
>> index 4c4aaf61dd3..b4b39579634 100644
>> --- a/tests/tcg/aarch64/Makefile.softmmu-target
>> +++ b/tests/tcg/aarch64/Makefile.softmmu-target
>> @@ -32,3 +32,24 @@ memory: CFLAGS+=-DCHECK_UNALIGNED=1
>>
>>  # Running
>>  QEMU_OPTS+=-M virt -cpu max -display none -semihosting-config
>> enable=on,target=native,chardev=output -kernel
>> +
>> +# Simple Record/Replay Test
>> +.PHONY: memory-record
>> +run-memory-record: memory-record memory
>> +	$(call run-test, $<, \
>> +	  $(QEMU) -monitor none -display none \
>> +		  -chardev file$(COMMA)path=$<.out$(COMMA)id=output \
>> +		  -icount shift=5$(COMMA)rr=record$(COMMA)rrfile=record.bin \
>> +	   	  $(QEMU_OPTS) memory, \
>> +	  "$< on $(TARGET_NAME)")
>> +
>> +.PHONY: memory-replay
>> +run-memory-replay: memory-replay run-memory-record
>> +	$(call run-test, $<, \
>> +	  $(QEMU) -monitor none -display none \
>> +		  -chardev file$(COMMA)path=$<.out$(COMMA)id=output \
>> +		  -icount shift=5$(COMMA)rr=replay$(COMMA)rrfile=record.bin \
>> +	   	  $(QEMU_OPTS) memory, \
>> +	  "$< on $(TARGET_NAME)")
>> +
>> +TESTS+=memory-record memory-replay
>> --
>> 2.20.1


--
Alex Bennée
[Qemu-devel] [RFC PATCH] tests/tcg: add time counter to memory test (WIP, aarch64 only)
Posted by Alex Bennée 4 years, 7 months ago 
To better support testing record/replay we should output something
that would otherwise be variable had it not been pegged to icount.

[AJB: needs a bit of work to nicely work across architectures although
most have some sort of counter].

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Cc: Pavel Dovgalyuk <dovgaluk@ispras.ru>
---
 tests/tcg/aarch64/Makefile.softmmu-target | 1 +
 tests/tcg/aarch64/system/boot.S           | 6 ++++++
 tests/tcg/minilib/minilib.h               | 3 +++
 tests/tcg/multiarch/system/memory.c       | 6 +++++-
 4 files changed, 15 insertions(+), 1 deletion(-)

diff --git a/tests/tcg/aarch64/Makefile.softmmu-target b/tests/tcg/aarch64/Makefile.softmmu-target
index b4b39579634..698d9b26b63 100644
--- a/tests/tcg/aarch64/Makefile.softmmu-target
+++ b/tests/tcg/aarch64/Makefile.softmmu-target
@@ -51,5 +51,6 @@ run-memory-replay: memory-replay run-memory-record
 		  -icount shift=5$(COMMA)rr=replay$(COMMA)rrfile=record.bin \
 	   	  $(QEMU_OPTS) memory, \
 	  "$< on $(TARGET_NAME)")
+	$(call diff-out,memory-record,memory-replay.out)
 
 TESTS+=memory-record memory-replay
diff --git a/tests/tcg/aarch64/system/boot.S b/tests/tcg/aarch64/system/boot.S
index b14e94f332d..fb7574e9c06 100644
--- a/tests/tcg/aarch64/system/boot.S
+++ b/tests/tcg/aarch64/system/boot.S
@@ -220,6 +220,12 @@ __sys_outc:
 	ldp x0, x1, [sp], #16
 	ret
 
+	/* return some sort of counter value */
+	.global __sys_counter
+__sys_counter:
+	mrs x0, CNTV_TVAL_EL0
+	ret
+
 	.data
 	.align	12
 
diff --git a/tests/tcg/minilib/minilib.h b/tests/tcg/minilib/minilib.h
index e23361380ae..29c097c545b 100644
--- a/tests/tcg/minilib/minilib.h
+++ b/tests/tcg/minilib/minilib.h
@@ -12,10 +12,13 @@
 #ifndef _MINILIB_H_
 #define _MINILIB_H_
 
+#include <inttypes.h>
+
 /*
  * Provided by the individual arch
  */
 extern void __sys_outc(char c);
+extern uint64_t __sys_counter(void);
 
 /*
  * Provided by the common minilib
diff --git a/tests/tcg/multiarch/system/memory.c b/tests/tcg/multiarch/system/memory.c
index d124502d730..fe32ed27bf8 100644
--- a/tests/tcg/multiarch/system/memory.c
+++ b/tests/tcg/multiarch/system/memory.c
@@ -454,6 +454,8 @@ int main(void)
 {
     int i;
     bool ok = true;
+    uint64_t start = __sys_counter();
+    uint64_t end;
 
     /* Run through the unsigned tests first */
     for (i = 0; i < ARRAY_SIZE(init_ufns) && ok; i++) {
@@ -470,6 +472,8 @@ int main(void)
         ok = do_signed_reads(true);
     }
 
-    ml_printf("Test complete: %s\n", ok ? "PASSED" : "FAILED");
+    end = __sys_counter();
+
+    ml_printf("Test complete in %" PRId64 ": %s\n", start - end, ok ? "PASSED" : "FAILED");
     return ok ? 0 : -1;
 }
-- 
2.20.1

Patchew 2.1-devel-0870f84

© 2016 - 2024 Red Hat, Inc.