1. Patchew
  2. Xen
  3. View series

[XEN PATCH] CI: Always move the bisect build log back

Anthony PERARD posted 1 patch 8 months, 2 weeks ago
Patches applied successfully (tree, apply log)
git fetch https://gitlab.com/xen-project/patchew/xen tags/patchew/20230823152334.8867-1-anthony.perard@citrix.com
automation/gitlab-ci/test.yaml | 1 +
1 file changed, 1 insertion(+)
[XEN PATCH] CI: Always move the bisect build log back
Posted by Anthony PERARD 8 months, 2 weeks ago 
On failure of "build"-each-commit script, the next command that move
the log back into the build directory isn't executed. Fix that by
using "after_script" which is always executed even if the main
"script" fails. (We would still miss the log when the jobs times out.)

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
---
 automation/gitlab-ci/test.yaml | 1 +
 1 file changed, 1 insertion(+)

diff --git a/automation/gitlab-ci/test.yaml b/automation/gitlab-ci/test.yaml
index 810631bc46..5099f2e6b6 100644
--- a/automation/gitlab-ci/test.yaml
+++ b/automation/gitlab-ci/test.yaml
@@ -140,6 +140,7 @@ build-each-commit-gcc:
     CC: gcc
   script:
     - BASE=${BASE_SHA:-${CI_COMMIT_BEFORE_SHA}} TIP=${TIP_SHA:-${CI_COMMIT_SHA}} ./automation/gitlab-ci/build-each-commit.sh 2>&1 | tee ../build-each-commit-gcc.log
+  after_script:
     - mv ../build-each-commit-gcc.log .
   artifacts:
     paths:
-- 
Anthony PERARD
Re: [XEN PATCH] CI: Always move the bisect build log back
Posted by Andrew Cooper 8 months, 2 weeks ago 
On 23/08/2023 4:23 pm, Anthony PERARD wrote:
> On failure of "build"-each-commit script, the next command that move
> the log back into the build directory isn't executed. Fix that by
> using "after_script" which is always executed even if the main
> "script" fails. (We would still miss the log when the jobs times out.)
>
> Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
> ---
>  automation/gitlab-ci/test.yaml | 1 +
>  1 file changed, 1 insertion(+)
>
> diff --git a/automation/gitlab-ci/test.yaml b/automation/gitlab-ci/test.yaml
> index 810631bc46..5099f2e6b6 100644
> --- a/automation/gitlab-ci/test.yaml
> +++ b/automation/gitlab-ci/test.yaml
> @@ -140,6 +140,7 @@ build-each-commit-gcc:
>      CC: gcc
>    script:
>      - BASE=${BASE_SHA:-${CI_COMMIT_BEFORE_SHA}} TIP=${TIP_SHA:-${CI_COMMIT_SHA}} ./automation/gitlab-ci/build-each-commit.sh 2>&1 | tee ../build-each-commit-gcc.log
> +  after_script:
>      - mv ../build-each-commit-gcc.log .
>    artifacts:
>      paths:

Thanks for looking into this, and yeah that is dumb, but why play games
with the parent directory?

$ git diff
diff --git a/automation/gitlab-ci/test.yaml b/automation/gitlab-ci/test.yaml
index 810631bc4624..b4c2f22a1b07 100644
--- a/automation/gitlab-ci/test.yaml
+++ b/automation/gitlab-ci/test.yaml
@@ -136,11 +136,11 @@ build-each-commit-gcc:
   extends: .test-jobs-common
   variables:
     CONTAINER: debian:stretch
+    LOGFILE: build-each-commit-gcc.log
     XEN_TARGET_ARCH: x86_64
     CC: gcc
   script:
-    - BASE=${BASE_SHA:-${CI_COMMIT_BEFORE_SHA}}
TIP=${TIP_SHA:-${CI_COMMIT_SHA}}
./automation/gitlab-ci/build-each-commit.sh 2>&1 | tee
../build-each-commit-gcc.log
-    - mv ../build-each-commit-gcc.log .
+    - BASE=${BASE_SHA:-${CI_COMMIT_BEFORE_SHA}}
TIP=${TIP_SHA:-${CI_COMMIT_SHA}}
./automation/gitlab-ci/build-each-commit.sh 2>&1 | tee ${LOGFILE}
   artifacts:
     paths:
       - '*.log'


This is prevailing style of the other tests, and also e.g. won't emit
the (whole) log file if e.g. disk space fills up.

~Andrew
Re: [XEN PATCH] CI: Always move the bisect build log back
Posted by Anthony PERARD 8 months, 2 weeks ago 
On Wed, Aug 23, 2023 at 07:05:56PM +0100, Andrew Cooper wrote:
> On 23/08/2023 4:23 pm, Anthony PERARD wrote:
> > On failure of "build"-each-commit script, the next command that move
> > the log back into the build directory isn't executed. Fix that by
> > using "after_script" which is always executed even if the main
> > "script" fails. (We would still miss the log when the jobs times out.)
> >
> > Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
> > ---
> >  automation/gitlab-ci/test.yaml | 1 +
> >  1 file changed, 1 insertion(+)
> >
> > diff --git a/automation/gitlab-ci/test.yaml b/automation/gitlab-ci/test.yaml
> > index 810631bc46..5099f2e6b6 100644
> > --- a/automation/gitlab-ci/test.yaml
> > +++ b/automation/gitlab-ci/test.yaml
> > @@ -140,6 +140,7 @@ build-each-commit-gcc:
> >      CC: gcc
> >    script:
> >      - BASE=${BASE_SHA:-${CI_COMMIT_BEFORE_SHA}} TIP=${TIP_SHA:-${CI_COMMIT_SHA}} ./automation/gitlab-ci/build-each-commit.sh 2>&1 | tee ../build-each-commit-gcc.log
> > +  after_script:
> >      - mv ../build-each-commit-gcc.log .
> >    artifacts:
> >      paths:
> 
> Thanks for looking into this, and yeah that is dumb, but why play games
> with the parent directory?

`git clean -ffdx` has the tendency to remove everything that's not
committed, that's why. But maybe we can teach ./build-each-commit.sh to
ignore that logfile.

Cheers,

-- 
Anthony PERARD
Re: [XEN PATCH] CI: Always move the bisect build log back
Posted by Andrew Cooper 8 months, 2 weeks ago 
On 24/08/2023 9:52 am, Anthony PERARD wrote:
> On Wed, Aug 23, 2023 at 07:05:56PM +0100, Andrew Cooper wrote:
>> On 23/08/2023 4:23 pm, Anthony PERARD wrote:
>>> On failure of "build"-each-commit script, the next command that move
>>> the log back into the build directory isn't executed. Fix that by
>>> using "after_script" which is always executed even if the main
>>> "script" fails. (We would still miss the log when the jobs times out.)
>>>
>>> Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
>>> ---
>>>  automation/gitlab-ci/test.yaml | 1 +
>>>  1 file changed, 1 insertion(+)
>>>
>>> diff --git a/automation/gitlab-ci/test.yaml b/automation/gitlab-ci/test.yaml
>>> index 810631bc46..5099f2e6b6 100644
>>> --- a/automation/gitlab-ci/test.yaml
>>> +++ b/automation/gitlab-ci/test.yaml
>>> @@ -140,6 +140,7 @@ build-each-commit-gcc:
>>>      CC: gcc
>>>    script:
>>>      - BASE=${BASE_SHA:-${CI_COMMIT_BEFORE_SHA}} TIP=${TIP_SHA:-${CI_COMMIT_SHA}} ./automation/gitlab-ci/build-each-commit.sh 2>&1 | tee ../build-each-commit-gcc.log
>>> +  after_script:
>>>      - mv ../build-each-commit-gcc.log .
>>>    artifacts:
>>>      paths:
>> Thanks for looking into this, and yeah that is dumb, but why play games
>> with the parent directory?
> `git clean -ffdx` has the tendency to remove everything that's not
> committed, that's why. But maybe we can teach ./build-each-commit.sh to
> ignore that logfile.

Oh, right.  Yeah, lets not lose the log file like that.

I'd say that teaching `git clean` to leave the file interacted and not
copying it is going to be a more robust option.

~Andrew
Re: [XEN PATCH] CI: Always move the bisect build log back
Posted by Anthony PERARD 8 months, 2 weeks ago 
On Thu, Aug 24, 2023 at 10:51:20AM +0100, Andrew Cooper wrote:
> On 24/08/2023 9:52 am, Anthony PERARD wrote:
> > On Wed, Aug 23, 2023 at 07:05:56PM +0100, Andrew Cooper wrote:
> >> On 23/08/2023 4:23 pm, Anthony PERARD wrote:
> >>> On failure of "build"-each-commit script, the next command that move
> >>> the log back into the build directory isn't executed. Fix that by
> >>> using "after_script" which is always executed even if the main
> >>> "script" fails. (We would still miss the log when the jobs times out.)
> >>>
> >>> Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
> >>> ---
> >>>  automation/gitlab-ci/test.yaml | 1 +
> >>>  1 file changed, 1 insertion(+)
> >>>
> >>> diff --git a/automation/gitlab-ci/test.yaml b/automation/gitlab-ci/test.yaml
> >>> index 810631bc46..5099f2e6b6 100644
> >>> --- a/automation/gitlab-ci/test.yaml
> >>> +++ b/automation/gitlab-ci/test.yaml
> >>> @@ -140,6 +140,7 @@ build-each-commit-gcc:
> >>>      CC: gcc
> >>>    script:
> >>>      - BASE=${BASE_SHA:-${CI_COMMIT_BEFORE_SHA}} TIP=${TIP_SHA:-${CI_COMMIT_SHA}} ./automation/gitlab-ci/build-each-commit.sh 2>&1 | tee ../build-each-commit-gcc.log
> >>> +  after_script:
> >>>      - mv ../build-each-commit-gcc.log .
> >>>    artifacts:
> >>>      paths:
> >> Thanks for looking into this, and yeah that is dumb, but why play games
> >> with the parent directory?
> > `git clean -ffdx` has the tendency to remove everything that's not
> > committed, that's why. But maybe we can teach ./build-each-commit.sh to
> > ignore that logfile.
> 
> Oh, right.  Yeah, lets not lose the log file like that.
> 
> I'd say that teaching `git clean` to leave the file interacted and not
> copying it is going to be a more robust option.

Yep, just tried that. But "Tree is dirty, aborted" :'(

./build-test.sh refuses to run if there's something in the git worktree.

This test is going to need more rework to be useful in the gitlab-ci.

-- 
Anthony PERARD
Re: [XEN PATCH] CI: Always move the bisect build log back
Posted by Andrew Cooper 8 months, 2 weeks ago 
On 24/08/2023 11:48 am, Anthony PERARD wrote:
> On Thu, Aug 24, 2023 at 10:51:20AM +0100, Andrew Cooper wrote:
>> On 24/08/2023 9:52 am, Anthony PERARD wrote:
>>> On Wed, Aug 23, 2023 at 07:05:56PM +0100, Andrew Cooper wrote:
>>>> On 23/08/2023 4:23 pm, Anthony PERARD wrote:
>>>>> On failure of "build"-each-commit script, the next command that move
>>>>> the log back into the build directory isn't executed. Fix that by
>>>>> using "after_script" which is always executed even if the main
>>>>> "script" fails. (We would still miss the log when the jobs times out.)
>>>>>
>>>>> Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
>>>>> ---
>>>>>  automation/gitlab-ci/test.yaml | 1 +
>>>>>  1 file changed, 1 insertion(+)
>>>>>
>>>>> diff --git a/automation/gitlab-ci/test.yaml b/automation/gitlab-ci/test.yaml
>>>>> index 810631bc46..5099f2e6b6 100644
>>>>> --- a/automation/gitlab-ci/test.yaml
>>>>> +++ b/automation/gitlab-ci/test.yaml
>>>>> @@ -140,6 +140,7 @@ build-each-commit-gcc:
>>>>>      CC: gcc
>>>>>    script:
>>>>>      - BASE=${BASE_SHA:-${CI_COMMIT_BEFORE_SHA}} TIP=${TIP_SHA:-${CI_COMMIT_SHA}} ./automation/gitlab-ci/build-each-commit.sh 2>&1 | tee ../build-each-commit-gcc.log
>>>>> +  after_script:
>>>>>      - mv ../build-each-commit-gcc.log .
>>>>>    artifacts:
>>>>>      paths:
>>>> Thanks for looking into this, and yeah that is dumb, but why play games
>>>> with the parent directory?
>>> `git clean -ffdx` has the tendency to remove everything that's not
>>> committed, that's why. But maybe we can teach ./build-each-commit.sh to
>>> ignore that logfile.
>> Oh, right.  Yeah, lets not lose the log file like that.
>>
>> I'd say that teaching `git clean` to leave the file interacted and not
>> copying it is going to be a more robust option.
> Yep, just tried that. But "Tree is dirty, aborted" :'(
>
> ./build-test.sh refuses to run if there's something in the git worktree.
>
> This test is going to need more rework to be useful in the gitlab-ci.

Urgh fine.  Lets just go with your fix in the short term.  It's
definitely better than nothing.

~Andrew

Patchew 2.1-devel-0870f84

© 2016 - 2024 Red Hat, Inc.