[v1] docker/shippable: cross-build mipsel and powerpc targets

[Qemu-devel] [PATCH 02/23] shippable: build using all available cpus

Posted by Philippe Mathieu-Daudé 8 years, 9 months ago

Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
---
 .shippable.yml | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/.shippable.yml b/.shippable.yml
index 231c29b620..5170486ff9 100644
--- a/.shippable.yml
+++ b/.shippable.yml
@@ -20,4 +20,4 @@ build:
   ci:
     - unset CC
     - ./configure ${QEMU_CONFIGURE_OPTS} --target-list=${TARGET_LIST}
-    - make -j2
+    - make -j$(getconf _NPROCESSORS_ONLN)
-- 
2.11.0

Re: [Qemu-devel] [PATCH 02/23] shippable: build using all available cpus

Posted by Alex Bennée 8 years, 9 months ago

Philippe Mathieu-Daudé <f4bug@amsat.org> writes:

> Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
> ---
>  .shippable.yml | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/.shippable.yml b/.shippable.yml
> index 231c29b620..5170486ff9 100644
> --- a/.shippable.yml
> +++ b/.shippable.yml
> @@ -20,4 +20,4 @@ build:
>    ci:
>      - unset CC
>      - ./configure ${QEMU_CONFIGURE_OPTS} --target-list=${TARGET_LIST}
> -    - make -j2
> +    - make -j$(getconf _NPROCESSORS_ONLN)

Did you ever check what this generates on Shippable? AFAICT they only
have one core for each docker image. I following the core+1 pattern
although that may not make as much sense for low core counts.

--
Alex Bennée

Re: [Qemu-devel] [PATCH 02/23] shippable: build using all available cpus

Posted by Philippe Mathieu-Daudé 8 years, 9 months ago

On 05/08/2017 07:55 AM, Alex Bennée wrote:
>
> Philippe Mathieu-Daudé <f4bug@amsat.org> writes:
>
>> Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
>> ---
>>  .shippable.yml | 2 +-
>>  1 file changed, 1 insertion(+), 1 deletion(-)
>>
>> diff --git a/.shippable.yml b/.shippable.yml
>> index 231c29b620..5170486ff9 100644
>> --- a/.shippable.yml
>> +++ b/.shippable.yml
>> @@ -20,4 +20,4 @@ build:
>>    ci:
>>      - unset CC
>>      - ./configure ${QEMU_CONFIGURE_OPTS} --target-list=${TARGET_LIST}
>> -    - make -j2
>> +    - make -j$(getconf _NPROCESSORS_ONLN)
>
> Did you ever check what this generates on Shippable? AFAICT they only
> have one core for each docker image. I following the core+1 pattern
> although that may not make as much sense for low core counts.

 From the 'Job node info':

$ sudo docker info
Architecture: x86_64
CPUs: 2

Checking at 'ci' stage:

$ echo "container proc:" `getconf _NPROCESSORS_ONLN` `getconf 
_NPROCESSORS_CONF`
container proc: 2 2

I think the correct cores+1 pattern should instead be:

make -j$(($(getconf _NPROCESSORS_ONLN) + 1))

So we stay building at full speed if Shippable eventually upgrade/allow 
to use more cores.

Phil.

> --
> Alex Bennée
>

Re: [Qemu-devel] [PATCH 02/23] shippable: build using all available cpus

Posted by Philippe Mathieu-Daudé 8 years, 9 months ago

>>> diff --git a/.shippable.yml b/.shippable.yml
>>> index 231c29b620..5170486ff9 100644
>>> --- a/.shippable.yml
>>> +++ b/.shippable.yml
>>> @@ -20,4 +20,4 @@ build:
>>>    ci:
>>>      - unset CC
>>>      - ./configure ${QEMU_CONFIGURE_OPTS} --target-list=${TARGET_LIST}
>>> -    - make -j2
>>> +    - make -j$(getconf _NPROCESSORS_ONLN)
>>
>> Did you ever check what this generates on Shippable? AFAICT they only
>> have one core for each docker image. I following the core+1 pattern
>> although that may not make as much sense for low core counts.
>
> From the 'Job node info':
>
> $ sudo docker info
> Architecture: x86_64
> CPUs: 2
>
> Checking at 'ci' stage:
>
> $ echo "container proc:" `getconf _NPROCESSORS_ONLN` `getconf
> _NPROCESSORS_CONF`
> container proc: 2 2
>
> I think the correct cores+1 pattern should instead be:
>
> make -j$(($(getconf _NPROCESSORS_ONLN) + 1))
>
> So we stay building at full speed if Shippable eventually upgrade/allow
> to use more cores.

For what it's worth using this pattern builds last on average <8min vs 
 >9min previously :)