[Fuego] [PATCH 5/9] Remove jenkins user created during build time with host's uid/gid

Guilherme Camargo guicc at profusion.mobi
Wed Jan 31 15:11:25 UTC 2018


Hello, Tim

Thanks for your input.

On Tue, Jan 30, 2018 at 6:39 PM, <Tim.Bird at sony.com> wrote:

> > -----Original Message-----
> > From: Guilherme Campos Camargo
> >
> > Prior to this patch, the fuego-host-scripts/docker-build-image.sh script
> > was calling docker build passing two `build-arg`s: uid/gid. According to
> > the comments in the code, it seems that the intention was to make sure
> > that the jenkins user (needed by Jenkins) would have the same uid/gid as
> > the user of the host that has called the install script.
> >
> > Given that the uid/gid of the jenkins user was being set at build time,
> > the generated image would be configured only for the machine where it's
> > built, what would prevent jenkins to change files in the /fuego-rw
> > mountpoint in different hosts.
>
> That's correct, and changing this is a nice step forward to making the
> container usable on other machines than the one it was built on.
>
> >
> > One approach for allowing jenkins to write to fuego-rw, is to chown
> > /fuego-rw and it's pre-existing subdirectories during execution time,
> > what can be done in a docker entrypoint script. That approach per se,
> > solves the problem, by allowing jenkins to write to the mount point, but
> > generates a side-effect: the user on the host machine will be able to
> > inspect, but will not be able to delete/modify the files created by
> > jenkins (unless it forces it as root).
> >
> > Another approach would be to change the uid and the gid of the jenkins
> > user in the first execution, (first call to docker-container-start.sh),
> > also through the entrypoint.  This would solve the problem of the access
> > from both sides, but has the downside of the need of running a recursive
> > `chown` in all jenkins files in `/var` (including cache), possibly
> > taking a long time to be accomplished.
>
> I'd much rather do this.  This amounts to a one-time container installation
> overhead on a new host, for the Fuego user.
>
> I modify files in fuego-ro and fuego-rw all the time, from the host side.
> This is my normal workflow, and I'd like to see it supported.
>
> >
> > Since the need to edit/remove the files from fuego-rw (without sudo)
> > depends on the user, our suggestion is to use the entrypoint for
> > `chowning` only the pre-existing directories from fuego-rw and to
> > provide the user a script for mapping the jenkins' uid/gid with user's
> > uid/gid only if the user needs to.
> >
> > On this patch we have created the entrypoint.sh script, and the uid/gid
> > mapping script.
>
> At this point I'm confused.  Do both approaches have an entrypoint.sh
> script?


> Could you give an example of the steps required to install the container
> in both your scenarios, including the arguments required for the scripts,
> and the resulting permissions and ownership both inside and outside
> the container, for the volumes?
>

With both approaches, there would be no change compared to the instructions
that
are given in Fuego's Quick Start guide.​

$ ./install.sh
$ ./fuego-host-scripts/create-container.sh
$ ./fuego-host-scripts/start-container.sh

We agree with you, and decided to move to the second approach
(set the uid/gid at the first start through the entrypoint).

Explaining:

With the first approach (the one that was implemented in the first version
of
my patch), the uid and the gid of the Jenkins user, inside the container,
would
be different than the uid/gid of the user that's running the container from
the
host. In order to allow Jenkins to write into fuego-rw, we were changing the
ownership of fuego-rw, fuego-rw/boards, fuego-rw/logs and
fuego-rw/buildzone to
the Jenkins user inside the container (not recursively). This would allow
jenkins to write to those directories and the change would propagate to the
outside, not allowing the user to modify the contents of fuego-rw anymore.

Apparently there was a misconception from our side, that most users would
not use fuego-rw to write, but only to read, and that Jenkins would be the
only
one (most of times) to write into that directory.

In order to allow those users to write into that directory as well, we had
added a new script (fuego-host-scripts/jenkins-map-uid.sh), that mapped
Jenkin's uid/gid to the host user's only on-demand.

In order to run that script the user would simply call without any arguments
$ ./fuego-host-scripts/jenkins-map-uid.sh

On docker, the only way to run a script inside the container is to have that
script copied to the container. Therefore,
fuego-hosts-scripts/jenkins-map-uid.sh was just calling that other script
(setup/jenkins/map-jenkins-uid-to-host.sh) that's been copied during Docker
build. "map-jenkins-uid-to-host.sh" is the one that would actually change
jenkins uid and chown the nedeed directories.

With the second approach (the one that we have implemented in our fixup),
the uid and the gid of the Jenkins user, inside the container, would be the
same as the
user that's running the container from the host. What means that, if the
owner
of the fuego-rw directory is the same user that's running the container,
fuego-rw will be accessible from both sides seamlessly.

The downside of this approach is that the create-container.sh step would
take
longer, due to the recursive chowns that are needed after replacing Jenkins'
uid/gid with the user's.

However, as you said, given that most of users will actually need to write
to
fuego-rw, the second approach is clearly the best, as you agree. So I'm
sending
a new patch, implementing the second approach, as a replacement for this
one.

I'm rebasing over the new 'next' branch as well.


> I'm not sure when map-jenkins-uid-to-host.sh and
> jenkins-map-uid.sh are run, and there seem to be no instructions
> for using them, or a human-readable description of what they do.
>
> Thanks,
>  -- Tim
>
> >
> > Signed-off-by: Guilherme Campos Camargo <guicc at profusion.mobi>
> > ---
> >  Dockerfile                                             | 10 ++++------
> >  frontend-install/entrypoint.sh                         | 13
> +++++++++++++
> >  .../setup/jenkins/map-jenkins-uid-to-host.sh           | 12
> ++++++++++++
> >  frontend-install/setup/jenkins/setup.sh                |  8 --------
> >  fuego-host-scripts/docker-build-image.sh               | 10 +---------
> >  fuego-host-scripts/jenkins-map-uid.sh                  | 18
> ++++++++++++++++++
> >  6 files changed, 48 insertions(+), 23 deletions(-)
> >  create mode 100755 frontend-install/entrypoint.sh
> >  create mode 100755 frontend-install/setup/jenkins/map-jenkins-uid-to-
> > host.sh
> >  create mode 100755 fuego-host-scripts/jenkins-map-uid.sh
> >
> > diff --git a/Dockerfile b/Dockerfile
> > index 883ac21..493414e 100644
> > --- a/Dockerfile
> > +++ b/Dockerfile
> > @@ -110,16 +110,12 @@ RUN echo deb http://emdebian.org/tools/debian/
> > jessie main > /etc/apt/sources.li
> >  # Download and Install Jenkins
> >  #
> > ==========================================================
> > ====================
> >
> > -ENV uid=1000
> > -ENV gid=${uid}
> >  ARG JENKINS_VERSION=2.32.1
> >  ARG JENKINS_SHA=bfc226aabe2bb089623772950c4cc13aee613af1
> >  ARG JENKINS_URL=https://pkg.jenkins.io/debian-
> > stable/binary/jenkins_${JENKINS_VERSION}_all.deb
> >  ENV JENKINS_HOME=/var/lib/jenkins
> >
> > -RUN groupadd -g ${gid} ${group} && \
> > -    useradd -l -m -d "${JENKINS_HOME}" -u ${uid} -g ${gid} -G sudo -s
> > /bin/bash ${user} && \
> > -    curl -L -O ${JENKINS_URL} && \
> > +RUN curl -L -O ${JENKINS_URL} && \
> >      echo "${JENKINS_SHA} jenkins_${JENKINS_VERSION}_all.deb" | sha1sum
> > -c - && \
> >      dpkg -i jenkins_${JENKINS_VERSION}_all.deb && \
> >      rm jenkins_${JENKINS_VERSION}_all.deb
> > @@ -156,4 +152,6 @@ COPY docs/fuego-docs.pdf
> > $JENKINS_HOME/userContent/docs/fuego-docs.pdf
> >  # Setup startup command
> >  #
> > ==========================================================
> > ====================
> >
> > -ENTRYPOINT service jenkins start && service netperf start && /bin/bash
> > +WORKDIR /
> > +COPY frontend-install/entrypoint.sh /
> > +ENTRYPOINT ["/entrypoint.sh"]
> > diff --git a/frontend-install/entrypoint.sh
> b/frontend-install/entrypoint.sh
> > new file mode 100755
> > index 0000000..281b0fe
> > --- /dev/null
> > +++ b/frontend-install/entrypoint.sh
> > @@ -0,0 +1,13 @@
> > +#!/bin/bash
> > +set -e
> > +
> > +service jenkins stop >> /dev/null
> > +
> > +mkdir -p fuego-rw/boards /fuego-rw/logs /fuego-rw/buildzone
> > +chown jenkins:jenkins \
> > +    fuego-rw fuego-rw/boards /fuego-rw/logs /fuego-rw/buildzone
> > +
> > +service jenkins start
> > +service netperf start
> > +
> > +exec /bin/bash
> > diff --git a/frontend-install/setup/jenkins/map-jenkins-uid-to-host.sh
> > b/frontend-install/setup/jenkins/map-jenkins-uid-to-host.sh
> > new file mode 100755
> > index 0000000..002f165
> > --- /dev/null
> > +++ b/frontend-install/
> ​​
> setup/jenkins/map-jenkins-uid-to-host.sh
>
> Who calls this?  When, and with what args?
>
> > @@ -0,0 +1,12 @@
> > +#!/bin/bash
> > +set -e
> > +
> > +service jenkins stop
> > +
> > +usermod -u "${jenkins_uid}" jenkins
> > +groupmod -g "${jenkins_gid}" jenkins
> > +
> > +chown -R "${jenkins_uid}":"${jenkins_gid}" \
> > +    /var/lib/jenkins /var/cache/jenkins /var/log/jenkins /fuego-rw
> > +
> > +service jenkins start
> > diff --git a/frontend-install/setup/jenkins/setup.sh b/frontend-
> > install/setup/jenkins/setup.sh
> > index c550862..11ad371 100755
> > --- a/frontend-install/setup/jenkins/setup.sh
> > +++ b/frontend-install/setup/jenkins/setup.sh
> > @@ -2,14 +2,6 @@
> >
> >  set -e
> >
> > -user=jenkins
> > -group=jenkins
> > -uid=1000
> > -gid=${uid}
> > -
> > -# groupadd -g ${gid} ${group}
> > -# useradd -l -m -d "${JENKINS_HOME}" -u ${uid} -g ${gid} -G sudo -s
> > /bin/bash ${user}
> > -
> >  cp config.xml jenkins.model.JenkinsLocationConfiguration.xml
> > ${JENKINS_HOME}
> >
> >  source /etc/default/jenkins
> > diff --git a/fuego-host-scripts/docker-build-image.sh b/fuego-host-
> > scripts/docker-build-image.sh
> > index 7c11b11..6550a98 100755
> > --- a/fuego-host-scripts/docker-build-image.sh
> > +++ b/fuego-host-scripts/docker-build-image.sh
> > @@ -2,12 +2,4 @@
> >  # $1 - name for the docker image (default: fuego)
> >  DOCKERIMAGE=${1:-fuego}
> >
> > -if [ "$(id -u)" == "0" ]; then
> > -     JENKINS_UID=$(id -u $SUDO_USER)
> > -     JENKINS_GID=$(id -g $SUDO_USER)
> > -else
> > -     JENKINS_UID=$(id -u $USER)
> > -     JENKINS_GID=$(id -g $USER)
> > -fi
> > -
> > -sudo docker build -t ${DOCKERIMAGE} --build-arg
> > HTTP_PROXY=$http_proxy --build-arg uid=$JENKINS_UID --build-arg
> > gid=$JENKINS_GID .
> > +sudo docker build -t ${DOCKERIMAGE} --build-arg
> > HTTP_PROXY=$http_proxy .
> > diff --git a/fuego-host-scripts/jenkins-map-uid.sh b/fuego-host-
> > scripts/jenkins-map-uid.sh
> > new file mode 100755
> > index 0000000..fad619e
> > --- /dev/null
> > +++ b/fuego-host-scripts/jenkins-map-uid.sh
>
> Who calls this?  When, and with what args?
>
> > @@ -0,0 +1,18 @@
> > +#!/bin/bash
> > +# $1 - name for the docker container (default: fuego-container)
> > +DOCKERCONTAINER=${1:-fuego-container}
> > +
> > +if [ "${UID}" == "0" ]; then
> > +     uid=$(id -u "${SUDO_USER}")
> > +     gid=$(id -g "${SUDO_USER}")
> > +else
> > +     uid="${UID}"
> > +     gid=$(id -g "${USER}")
> > +fi
> > +
> > +sudo docker start "${DOCKERCONTAINER}" || \
> > +  echo "Please create Fuego docker container via docker-create-
> > container.sh script"
> > +
> > +sudo docker exec -e jenkins_uid="${uid}" -e jenkins_gid="${gid}" \
> > +    "${DOCKERCONTAINER}" \
> > +    /setup/jenkins/map-jenkins-uid-to-host.sh
> > --
> > 2.15.1
>
>
> I need to understand this better before I apply it.
>  -- Tim
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxfoundation.org/pipermail/fuego/attachments/20180131/3a8d9ed9/attachment-0001.html>


More information about the Fuego mailing list