[libvirt] [dockerfiles PATCH 2/3] refresh: Add libosinfo-related projects

Andrea Bolognani posted 3 patches 6 years, 6 months ago
[libvirt] [dockerfiles PATCH 2/3] refresh: Add libosinfo-related projects
Posted by Andrea Bolognani 6 years, 6 months ago
The libosinfo project would like to start using these container
images in their own CI pipeline, so they need the corresponding
build dependencies to be included.

Signed-off-by: Andrea Bolognani <abologna@redhat.com>
---
 refresh | 5 +++++
 1 file changed, 5 insertions(+)

diff --git a/refresh b/refresh
index 81df77e..2491613 100755
--- a/refresh
+++ b/refresh
@@ -59,13 +59,18 @@ class Dockerfile:
             self.cross_arch = None
 
         self.projects = [
+            "libosinfo",
             "libvirt",
+            "osinfo-db",
+            "osinfo-db-tools",
         ]
 
         # Fedora Rawhide is special in that we use it to perform MinGW
         # builds, so we need to add the corresponding projects
         if self.os == "fedora-rawhide":
             self.projects += [
+                "osinfo-db-tools+mingw*",
+                "libosinfo+mingw*",
                 "libvirt+mingw*",
             ]
 
-- 
2.21.0

--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list
Re: [libvirt] [dockerfiles PATCH 2/3] refresh: Add libosinfo-related projects
Posted by Fabiano Fidêncio 6 years, 6 months ago
On Thu, 2019-07-18 at 14:20 +0200, Andrea Bolognani wrote:
> The libosinfo project would like to start using these container
> images in their own CI pipeline, so they need the corresponding
> build dependencies to be included.
> 
> Signed-off-by: Andrea Bolognani <abologna@redhat.com>
> ---
>  refresh | 5 +++++
>  1 file changed, 5 insertions(+)
> 
> diff --git a/refresh b/refresh
> index 81df77e..2491613 100755
> --- a/refresh
> +++ b/refresh
> @@ -59,13 +59,18 @@ class Dockerfile:
>              self.cross_arch = None
>  
>          self.projects = [
> +            "libosinfo",
>              "libvirt",
> +            "osinfo-db",
> +            "osinfo-db-tools",
>          ]
>  
>          # Fedora Rawhide is special in that we use it to perform
> MinGW
>          # builds, so we need to add the corresponding projects
>          if self.os == "fedora-rawhide":
>              self.projects += [
> +                "osinfo-db-tools+mingw*",

Please, move osinfo-db-tools+mingw* after libvirt+mingw* entry. By
doing this, we'll avoid triggering your OCD. 0:-)


> +                "libosinfo+mingw*",
>                  "libvirt+mingw*",
>              ]
>  

--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list
Re: [libvirt] [dockerfiles PATCH 2/3] refresh: Add libosinfo-related projects
Posted by Daniel P. Berrangé 6 years, 6 months ago
On Thu, Jul 18, 2019 at 02:20:34PM +0200, Andrea Bolognani wrote:
> The libosinfo project would like to start using these container
> images in their own CI pipeline, so they need the corresponding
> build dependencies to be included.
> 
> Signed-off-by: Andrea Bolognani <abologna@redhat.com>
> ---
>  refresh | 5 +++++
>  1 file changed, 5 insertions(+)
> 
> diff --git a/refresh b/refresh
> index 81df77e..2491613 100755
> --- a/refresh
> +++ b/refresh
> @@ -59,13 +59,18 @@ class Dockerfile:
>              self.cross_arch = None
>  
>          self.projects = [
> +            "libosinfo",
>              "libvirt",
> +            "osinfo-db",
> +            "osinfo-db-tools",
>          ]

We get away with this single container for libosinfo we don't have
any dependancies between components. It doesn't work in general
though.

ie the build deps for libvirt-perl skip 'libvirt' because they
assume we already chain built libvirt in the CI. This doesn't
happen except in Jenkins CI though.

We've never created a solution for fully populating deps such
that the system is self-contained and doesn't need to chain
build other projects first.

Regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|

--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list
Re: [libvirt] [dockerfiles PATCH 2/3] refresh: Add libosinfo-related projects
Posted by Andrea Bolognani 6 years, 6 months ago
On Thu, 2019-07-18 at 13:35 +0100, Daniel P. Berrangé wrote:
> On Thu, Jul 18, 2019 at 02:20:34PM +0200, Andrea Bolognani wrote:
> >          self.projects = [
> > +            "libosinfo",
> >              "libvirt",
> > +            "osinfo-db",
> > +            "osinfo-db-tools",
> >          ]
> 
> We get away with this single container for libosinfo we don't have
> any dependancies between components. It doesn't work in general
> though.
> 
> ie the build deps for libvirt-perl skip 'libvirt' because they
> assume we already chain built libvirt in the CI. This doesn't
> happen except in Jenkins CI though.

Yeah, that's a good point.

> We've never created a solution for fully populating deps such
> that the system is self-contained and doesn't need to chain
> build other projects first.

I guess we would need basically an alternative mode where you tell
lcitool to make the resulting guest or container image entirely
self-contained... And in order to do that we'd also need to teach
it about relationships between projects, which is something that up
until now we have managed to live without.

Even once we have that in place, though, the two modes can't quite
live side by side in the same container, so a project like for
example virt-manager will not be able to use the libvirt container
images but would have to prepare its own (using self-contained
mode)...

So perhaps we should not even consider merging this even though we
could still get away with it in this specific case, and instead have
the libosinfo project set up their own set of tailored container
images from day one?

-- 
Andrea Bolognani / Red Hat / Virtualization

--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list
Re: [libvirt] [dockerfiles PATCH 2/3] refresh: Add libosinfo-related projects
Posted by Daniel P. Berrangé 6 years, 6 months ago
On Thu, Jul 18, 2019 at 03:05:16PM +0200, Andrea Bolognani wrote:
> On Thu, 2019-07-18 at 13:35 +0100, Daniel P. Berrangé wrote:
> > On Thu, Jul 18, 2019 at 02:20:34PM +0200, Andrea Bolognani wrote:
> > >          self.projects = [
> > > +            "libosinfo",
> > >              "libvirt",
> > > +            "osinfo-db",
> > > +            "osinfo-db-tools",
> > >          ]
> > 
> > We get away with this single container for libosinfo we don't have
> > any dependancies between components. It doesn't work in general
> > though.
> > 
> > ie the build deps for libvirt-perl skip 'libvirt' because they
> > assume we already chain built libvirt in the CI. This doesn't
> > happen except in Jenkins CI though.
> 
> Yeah, that's a good point.

I probably shouldn't have picked the perl bindings on reflection,
because they can't be built without the libvirt from git.

Python / go bindings would be better as they aim to keep buildable
with historic versions.

> > We've never created a solution for fully populating deps such
> > that the system is self-contained and doesn't need to chain
> > build other projects first.
> 
> I guess we would need basically an alternative mode where you tell
> lcitool to make the resulting guest or container image entirely
> self-contained... And in order to do that we'd also need to teach
> it about relationships between projects, which is something that up
> until now we have managed to live without.

We don't need to teach it about relationships between projects
in an explicit sense, in fact it could be helpful if we don't
teach that. Rather we just need two lists of package. One minimal
set for the chain build scenario, and one extra set for the
self-contanied build scenario.

eg

---
chain_build_packages:
  - python2
  - python2-devel
  - python2-lxml
  - python2-nose
  - python3
  - python3-devel
  - python3-lxml
  - python3-nose
isolated_build_packages:
  - libvirt

Doing it with package lists would give us flexibility. For example,
for python where we want to test against many libvirt versions, we
have the opiton to list many versions using my virt-ark repo for
example

---
chain_build_packages:
  - python2
  - python2-devel
  - python2-lxml
  - python2-nose
  - python3
  - python3-devel
  - python3-lxml
  - python3-nose
isolated_build_packages:
  - libvirt-ark-2_0_0
  - libvirt-ark-3_0_0
  - libvirt-ark-4_0_0


> Even once we have that in place, though, the two modes can't quite
> live side by side in the same container, so a project like for
> example virt-manager will not be able to use the libvirt container
> images but would have to prepare its own (using self-contained
> mode)...
> 
> So perhaps we should not even consider merging this even though we
> could still get away with it in this specific case, and instead have
> the libosinfo project set up their own set of tailored container
> images from day one?

Having containers per project looks reasonable, especially as we can
now more easily automate their creation so it isn't such an admin
burden as before.

Probably makes sense to still keep it all under the libvirt org,
since we're still reusing the libvirt jenkins toolchain.

We'd just need a project specific name prefix on each

Regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|

--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list