1. Patchew
  2. QEMU
  3. [Qemu-devel] [PATCH v3 0/3] Docker cross-compile targets and user build support
  4. View patch

[Qemu-devel] [PATCH v3 1/3] tests/docker: add basic user mapping support

Alex Bennée posted 3 patches 8 years, 12 months ago
There is a newer version of this series
[Qemu-devel] [PATCH v3 1/3] tests/docker: add basic user mapping support
Posted by Alex Bennée 8 years, 12 months ago 
Currently all docker builds are done by exporting a tarball to the
docker container and running the build as the containers root user.
Other use cases are possible however and it is possible to map a part
of users file-system to the container. This is useful for example for
doing cross-builds of arbitrary source trees. For this to work
smoothly the container needs to have a user created that maps cleanly
to the host system.

This adds a -u option to the docker script so that:

  DEB_ARCH=armhf DEB_TYPE=stable ./tests/docker/docker.py build \
    -u --include-executable=arm-linux-user/qemu-arm \
    debian:armhf ./tests/docker/dockerfiles/debian-bootstrap.docker

Will build a container that can then be run like:

  docker run --rm -it -v /home/alex/lsrc/qemu/risu.git/:/src \
    --user=alex:alex -w /src/ debian:armhf \
    sh -c "make clean && ./configure -s && make"

Alternatively specifying USER will add the appropriate stanza when
building via the Makefile:

  make docker-image-debian-armhf-cross USER=1

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Fam Zheng <famz@redhat.com>

---
v2
  - write the useradd directly
  - change long option to --add-current-user
v3
  - images -> image's
  - add r-b
  - add USER to Makefile
---
 tests/docker/Makefile.include |  2 ++
 tests/docker/docker.py        | 15 +++++++++++++--
 2 files changed, 15 insertions(+), 2 deletions(-)

diff --git a/tests/docker/Makefile.include b/tests/docker/Makefile.include
index 3f15d5aea8..8db55e43aa 100644
--- a/tests/docker/Makefile.include
+++ b/tests/docker/Makefile.include
@@ -50,6 +50,7 @@ docker-image-%: $(DOCKER_FILES_DIR)/%.docker
 	$(call quiet-command,\
 		$(SRC_PATH)/tests/docker/docker.py build qemu:$* $< \
 		$(if $V,,--quiet) $(if $(NOCACHE),--no-cache) \
+		$(if $(USER),--add-current-user,) \
 		$(if $(EXECUTABLE),--include-executable=$(EXECUTABLE)),\
 		"BUILD","$*")
 
@@ -99,6 +100,7 @@ docker:
 	@echo '                         (default is 1)'
 	@echo '    DEBUG=1              Stop and drop to shell in the created container'
 	@echo '                         before running the command.'
+	@echo '    USER=1		Add current user to docker image.'
 	@echo '    NOCACHE=1            Ignore cache when build images.'
 	@echo '    EXECUTABLE=<path>    Include executable in image.'
 
diff --git a/tests/docker/docker.py b/tests/docker/docker.py
index 37d83199e7..3e6e7c6460 100755
--- a/tests/docker/docker.py
+++ b/tests/docker/docker.py
@@ -25,6 +25,7 @@ import signal
 from tarfile import TarFile, TarInfo
 from StringIO import StringIO
 from shutil import copy, rmtree
+from pwd import getpwuid
 
 
 DEVNULL = open(os.devnull, 'wb')
@@ -149,13 +150,20 @@ class Docker(object):
         labels = json.loads(resp)[0]["Config"].get("Labels", {})
         return labels.get("com.qemu.dockerfile-checksum", "")
 
-    def build_image(self, tag, docker_dir, dockerfile, quiet=True, argv=None):
+    def build_image(self, tag, docker_dir, dockerfile,
+                    quiet=True, user=False, argv=None):
         if argv == None:
             argv = []
 
         tmp_df = tempfile.NamedTemporaryFile(dir=docker_dir, suffix=".docker")
         tmp_df.write(dockerfile)
 
+        if user:
+            uid = os.getuid()
+            uname = getpwuid(uid).pw_name
+            tmp_df.write("\n")
+            tmp_df.write("RUN useradd -u %d -U %s" % (uid, uname))
+
         tmp_df.write("\n")
         tmp_df.write("LABEL com.qemu.dockerfile-checksum=%s" %
                      _text_checksum(dockerfile))
@@ -225,6 +233,9 @@ class BuildCommand(SubCommand):
                             help="""Specify a binary that will be copied to the
                             container together with all its dependent
                             libraries""")
+        parser.add_argument("--add-current-user", "-u", dest="user",
+                            action="store_true",
+                            help="Add the current user to image's passwd")
         parser.add_argument("tag",
                             help="Image Tag")
         parser.add_argument("dockerfile",
@@ -261,7 +272,7 @@ class BuildCommand(SubCommand):
                                        docker_dir)
 
             dkr.build_image(tag, docker_dir, dockerfile,
-                            quiet=args.quiet, argv=argv)
+                            quiet=args.quiet, user=args.user, argv=argv)
 
             rmtree(docker_dir)
 
-- 
2.11.0
Re: [Qemu-devel] [PATCH v3 1/3] tests/docker: add basic user mapping support
Posted by Fam Zheng 8 years, 12 months ago 
On Tue, 02/14 10:07, Alex Bennée wrote:
> Currently all docker builds are done by exporting a tarball to the
> docker container and running the build as the containers root user.
> Other use cases are possible however and it is possible to map a part
> of users file-system to the container. This is useful for example for
> doing cross-builds of arbitrary source trees. For this to work
> smoothly the container needs to have a user created that maps cleanly
> to the host system.
> 
> This adds a -u option to the docker script so that:
> 
>   DEB_ARCH=armhf DEB_TYPE=stable ./tests/docker/docker.py build \
>     -u --include-executable=arm-linux-user/qemu-arm \
>     debian:armhf ./tests/docker/dockerfiles/debian-bootstrap.docker
> 
> Will build a container that can then be run like:
> 
>   docker run --rm -it -v /home/alex/lsrc/qemu/risu.git/:/src \
>     --user=alex:alex -w /src/ debian:armhf \
>     sh -c "make clean && ./configure -s && make"
> 
> Alternatively specifying USER will add the appropriate stanza when
> building via the Makefile:
> 
>   make docker-image-debian-armhf-cross USER=1
> 
> Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
> Reviewed-by: Fam Zheng <famz@redhat.com>
> 
> ---
> v2
>   - write the useradd directly
>   - change long option to --add-current-user
> v3
>   - images -> image's
>   - add r-b
>   - add USER to Makefile
> ---
>  tests/docker/Makefile.include |  2 ++
>  tests/docker/docker.py        | 15 +++++++++++++--
>  2 files changed, 15 insertions(+), 2 deletions(-)
> 
> diff --git a/tests/docker/Makefile.include b/tests/docker/Makefile.include
> index 3f15d5aea8..8db55e43aa 100644
> --- a/tests/docker/Makefile.include
> +++ b/tests/docker/Makefile.include
> @@ -50,6 +50,7 @@ docker-image-%: $(DOCKER_FILES_DIR)/%.docker
>  	$(call quiet-command,\
>  		$(SRC_PATH)/tests/docker/docker.py build qemu:$* $< \
>  		$(if $V,,--quiet) $(if $(NOCACHE),--no-cache) \
> +		$(if $(USER),--add-current-user,) \
>  		$(if $(EXECUTABLE),--include-executable=$(EXECUTABLE)),\
>  		"BUILD","$*")
>  
> @@ -99,6 +100,7 @@ docker:
>  	@echo '                         (default is 1)'
>  	@echo '    DEBUG=1              Stop and drop to shell in the created container'
>  	@echo '                         before running the command.'
> +	@echo '    USER=1		Add current user to docker image.'

This is a bad choice of name, $USER is "$(id -un)", maybe ADD_CUR_USER?

Fam


>  	@echo '    NOCACHE=1            Ignore cache when build images.'
>  	@echo '    EXECUTABLE=<path>    Include executable in image.'
>
Re: [Qemu-devel] [PATCH v3 1/3] tests/docker: add basic user mapping support
Posted by Alex Bennée 8 years, 12 months ago 
Fam Zheng <famz@redhat.com> writes:

> On Tue, 02/14 10:07, Alex Bennée wrote:
>> Currently all docker builds are done by exporting a tarball to the
>> docker container and running the build as the containers root user.
>> Other use cases are possible however and it is possible to map a part
>> of users file-system to the container. This is useful for example for
>> doing cross-builds of arbitrary source trees. For this to work
>> smoothly the container needs to have a user created that maps cleanly
>> to the host system.
>>
>> This adds a -u option to the docker script so that:
>>
>>   DEB_ARCH=armhf DEB_TYPE=stable ./tests/docker/docker.py build \
>>     -u --include-executable=arm-linux-user/qemu-arm \
>>     debian:armhf ./tests/docker/dockerfiles/debian-bootstrap.docker
>>
>> Will build a container that can then be run like:
>>
>>   docker run --rm -it -v /home/alex/lsrc/qemu/risu.git/:/src \
>>     --user=alex:alex -w /src/ debian:armhf \
>>     sh -c "make clean && ./configure -s && make"
>>
>> Alternatively specifying USER will add the appropriate stanza when
>> building via the Makefile:
>>
>>   make docker-image-debian-armhf-cross USER=1
>>
>> Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
>> Reviewed-by: Fam Zheng <famz@redhat.com>
>>
>> ---
>> v2
>>   - write the useradd directly
>>   - change long option to --add-current-user
>> v3
>>   - images -> image's
>>   - add r-b
>>   - add USER to Makefile
>> ---
>>  tests/docker/Makefile.include |  2 ++
>>  tests/docker/docker.py        | 15 +++++++++++++--
>>  2 files changed, 15 insertions(+), 2 deletions(-)
>>
>> diff --git a/tests/docker/Makefile.include b/tests/docker/Makefile.include
>> index 3f15d5aea8..8db55e43aa 100644
>> --- a/tests/docker/Makefile.include
>> +++ b/tests/docker/Makefile.include
>> @@ -50,6 +50,7 @@ docker-image-%: $(DOCKER_FILES_DIR)/%.docker
>>  	$(call quiet-command,\
>>  		$(SRC_PATH)/tests/docker/docker.py build qemu:$* $< \
>>  		$(if $V,,--quiet) $(if $(NOCACHE),--no-cache) \
>> +		$(if $(USER),--add-current-user,) \
>>  		$(if $(EXECUTABLE),--include-executable=$(EXECUTABLE)),\
>>  		"BUILD","$*")
>>
>> @@ -99,6 +100,7 @@ docker:
>>  	@echo '                         (default is 1)'
>>  	@echo '    DEBUG=1              Stop and drop to shell in the created container'
>>  	@echo '                         before running the command.'
>> +	@echo '    USER=1		Add current user to docker image.'
>
> This is a bad choice of name, $USER is "$(id -un)", maybe
> ADD_CUR_USER?

Or we could just enable it by default? Is there a reason not to enable
the current user for these images?

>
> Fam
>
>
>>  	@echo '    NOCACHE=1            Ignore cache when build images.'
>>  	@echo '    EXECUTABLE=<path>    Include executable in image.'
>>


--
Alex Bennée
Re: [Qemu-devel] [PATCH v3 1/3] tests/docker: add basic user mapping support
Posted by Fam Zheng 8 years, 12 months ago 
On Tue, 02/14 15:11, Alex Bennée wrote:
> 
> Fam Zheng <famz@redhat.com> writes:
> 
> > On Tue, 02/14 10:07, Alex Bennée wrote:
> >> Currently all docker builds are done by exporting a tarball to the
> >> docker container and running the build as the containers root user.
> >> Other use cases are possible however and it is possible to map a part
> >> of users file-system to the container. This is useful for example for
> >> doing cross-builds of arbitrary source trees. For this to work
> >> smoothly the container needs to have a user created that maps cleanly
> >> to the host system.
> >>
> >> This adds a -u option to the docker script so that:
> >>
> >>   DEB_ARCH=armhf DEB_TYPE=stable ./tests/docker/docker.py build \
> >>     -u --include-executable=arm-linux-user/qemu-arm \
> >>     debian:armhf ./tests/docker/dockerfiles/debian-bootstrap.docker
> >>
> >> Will build a container that can then be run like:
> >>
> >>   docker run --rm -it -v /home/alex/lsrc/qemu/risu.git/:/src \
> >>     --user=alex:alex -w /src/ debian:armhf \
> >>     sh -c "make clean && ./configure -s && make"
> >>
> >> Alternatively specifying USER will add the appropriate stanza when
> >> building via the Makefile:
> >>
> >>   make docker-image-debian-armhf-cross USER=1
> >>
> >> Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
> >> Reviewed-by: Fam Zheng <famz@redhat.com>
> >>
> >> ---
> >> v2
> >>   - write the useradd directly
> >>   - change long option to --add-current-user
> >> v3
> >>   - images -> image's
> >>   - add r-b
> >>   - add USER to Makefile
> >> ---
> >>  tests/docker/Makefile.include |  2 ++
> >>  tests/docker/docker.py        | 15 +++++++++++++--
> >>  2 files changed, 15 insertions(+), 2 deletions(-)
> >>
> >> diff --git a/tests/docker/Makefile.include b/tests/docker/Makefile.include
> >> index 3f15d5aea8..8db55e43aa 100644
> >> --- a/tests/docker/Makefile.include
> >> +++ b/tests/docker/Makefile.include
> >> @@ -50,6 +50,7 @@ docker-image-%: $(DOCKER_FILES_DIR)/%.docker
> >>  	$(call quiet-command,\
> >>  		$(SRC_PATH)/tests/docker/docker.py build qemu:$* $< \
> >>  		$(if $V,,--quiet) $(if $(NOCACHE),--no-cache) \
> >> +		$(if $(USER),--add-current-user,) \
> >>  		$(if $(EXECUTABLE),--include-executable=$(EXECUTABLE)),\
> >>  		"BUILD","$*")
> >>
> >> @@ -99,6 +100,7 @@ docker:
> >>  	@echo '                         (default is 1)'
> >>  	@echo '    DEBUG=1              Stop and drop to shell in the created container'
> >>  	@echo '                         before running the command.'
> >> +	@echo '    USER=1		Add current user to docker image.'
> >
> > This is a bad choice of name, $USER is "$(id -un)", maybe
> > ADD_CUR_USER?
> 
> Or we could just enable it by default? Is there a reason not to enable
> the current user for these images?

I think that's fine, but then docker.py needs to tolerate user already existing
in the image, such as when you build qemu:debian-cross-$foo on top of
qemu:debian.

Fam

Patchew 2.1-devel-b67e4c2

© 2016 - 2026 Red Hat, Inc.