[PATCH] .gitlab-ci.d/buildtest.yml: Increase the check-gprof-gcov job timeout

Bin Meng posted 1 patch 1 year, 8 months ago
Patches applied successfully (tree, apply log)
git fetch https://github.com/patchew-project/qemu tags/patchew/20220817034630.1600434-1-bmeng.cn@gmail.com
Maintainers: "Alex Bennée" <alex.bennee@linaro.org>, "Philippe Mathieu-Daudé" <f4bug@amsat.org>, Thomas Huth <thuth@redhat.com>, Wainer dos Santos Moschetta <wainersm@redhat.com>, Beraldo Leal <bleal@redhat.com>
.gitlab-ci.d/buildtest.yml | 1 +
1 file changed, 1 insertion(+)
[PATCH] .gitlab-ci.d/buildtest.yml: Increase the check-gprof-gcov job timeout
Posted by Bin Meng 1 year, 8 months ago
Current project timeout is 1 hour, but the check-gprof-gcov job never
completes within 1 hour. Increase the job timeout to 90 minutes.

Signed-off-by: Bin Meng <bmeng.cn@gmail.com>
---

 .gitlab-ci.d/buildtest.yml | 1 +
 1 file changed, 1 insertion(+)

diff --git a/.gitlab-ci.d/buildtest.yml b/.gitlab-ci.d/buildtest.yml
index 1931b77b49..52d45508fb 100644
--- a/.gitlab-ci.d/buildtest.yml
+++ b/.gitlab-ci.d/buildtest.yml
@@ -495,6 +495,7 @@ check-gprof-gcov:
   variables:
     IMAGE: ubuntu2004
     MAKE_CHECK_ARGS: check
+  timeout: 90m
   after_script:
     - ${CI_PROJECT_DIR}/scripts/ci/coverage-summary.sh
 
-- 
2.34.1
Re: [PATCH] .gitlab-ci.d/buildtest.yml: Increase the check-gprof-gcov job timeout
Posted by Alex Bennée 1 year, 8 months ago
Bin Meng <bmeng.cn@gmail.com> writes:

> Current project timeout is 1 hour, but the check-gprof-gcov job never
> completes within 1 hour. Increase the job timeout to 90 minutes.

I've tried to do some data mining on this test which seems to indicate
that the average successful completion time for gprof-gcov is 27 mins.
So it seems the sometimes *something* causes it to run a lot more
slowly. I wonder if increasing the timeout just masks the problem?

>
> Signed-off-by: Bin Meng <bmeng.cn@gmail.com>
> ---
>
>  .gitlab-ci.d/buildtest.yml | 1 +
>  1 file changed, 1 insertion(+)
>
> diff --git a/.gitlab-ci.d/buildtest.yml b/.gitlab-ci.d/buildtest.yml
> index 1931b77b49..52d45508fb 100644
> --- a/.gitlab-ci.d/buildtest.yml
> +++ b/.gitlab-ci.d/buildtest.yml
> @@ -495,6 +495,7 @@ check-gprof-gcov:
>    variables:
>      IMAGE: ubuntu2004
>      MAKE_CHECK_ARGS: check
> +  timeout: 90m
>    after_script:
>      - ${CI_PROJECT_DIR}/scripts/ci/coverage-summary.sh


-- 
Alex Bennée
Re: [PATCH] .gitlab-ci.d/buildtest.yml: Increase the check-gprof-gcov job timeout
Posted by Thomas Huth 1 year, 8 months ago
On 18/08/2022 09.07, Alex Bennée wrote:
> 
> Bin Meng <bmeng.cn@gmail.com> writes:
> 
>> Current project timeout is 1 hour, but the check-gprof-gcov job never
>> completes within 1 hour. Increase the job timeout to 90 minutes.
> 
> I've tried to do some data mining on this test which seems to indicate
> that the average successful completion time for gprof-gcov is 27 mins.
> So it seems the sometimes *something* causes it to run a lot more
> slowly. I wonder if increasing the timeout just masks the problem?

I think it's very certainly the migration test that is causing the timeout 
now. In former times, the test was skipped, see e.g.:

  https://gitlab.com/qemu-project/qemu/-/jobs/2795092179#L48

Since commit 2649a72555e the tests are run and take ca. 2 * 20 minutes:

  https://gitlab.com/qemu-project/qemu/-/jobs/2798625330#L48

That's why I started a discussion here for how to reduce the amount of time 
of the migration tests in the CI:

  https://lists.gnu.org/archive/html/qemu-devel/2022-08/msg01219.html

  Thomas


Re: [PATCH] .gitlab-ci.d/buildtest.yml: Increase the check-gprof-gcov job timeout
Posted by Alex Bennée 1 year, 8 months ago
Thomas Huth <thuth@redhat.com> writes:

> On 18/08/2022 09.07, Alex Bennée wrote:
>> Bin Meng <bmeng.cn@gmail.com> writes:
>> 
>>> Current project timeout is 1 hour, but the check-gprof-gcov job never
>>> completes within 1 hour. Increase the job timeout to 90 minutes.
>> I've tried to do some data mining on this test which seems to
>> indicate
>> that the average successful completion time for gprof-gcov is 27 mins.
>> So it seems the sometimes *something* causes it to run a lot more
>> slowly. I wonder if increasing the timeout just masks the problem?
>
> I think it's very certainly the migration test that is causing the
> timeout now. In former times, the test was skipped, see e.g.:
>
>  https://gitlab.com/qemu-project/qemu/-/jobs/2795092179#L48
>
> Since commit 2649a72555e the tests are run and take ca. 2 * 20 minutes:
>
>  https://gitlab.com/qemu-project/qemu/-/jobs/2798625330#L48
>
> That's why I started a discussion here for how to reduce the amount of
> time of the migration tests in the CI:
>
>  https://lists.gnu.org/archive/html/qemu-devel/2022-08/msg01219.html

OK I have queued to this patch to testing/next while the discussion is
had. I'll update the commit message.

-- 
Alex Bennée
Re: [PATCH] .gitlab-ci.d/buildtest.yml: Increase the check-gprof-gcov job timeout
Posted by Thomas Huth 1 year, 8 months ago
On 17/08/2022 05.46, Bin Meng wrote:
> Current project timeout is 1 hour, but the check-gprof-gcov job never
> completes within 1 hour. Increase the job timeout to 90 minutes.
> 
> Signed-off-by: Bin Meng <bmeng.cn@gmail.com>
> ---
> 
>   .gitlab-ci.d/buildtest.yml | 1 +
>   1 file changed, 1 insertion(+)
> 
> diff --git a/.gitlab-ci.d/buildtest.yml b/.gitlab-ci.d/buildtest.yml
> index 1931b77b49..52d45508fb 100644
> --- a/.gitlab-ci.d/buildtest.yml
> +++ b/.gitlab-ci.d/buildtest.yml
> @@ -495,6 +495,7 @@ check-gprof-gcov:
>     variables:
>       IMAGE: ubuntu2004
>       MAKE_CHECK_ARGS: check
> +  timeout: 90m
>     after_script:
>       - ${CI_PROJECT_DIR}/scripts/ci/coverage-summary.sh

Thanks, I think we could use this as a workaround ... for tackling the real 
problem, we have to figure out a way to speed up the migration tests in the 
CI, see:

  https://lists.gnu.org/archive/html/qemu-devel/2022-08/msg01219.html

  Thomas