[Fuego] release test issues
Guilherme Camargo
guicc at profusion.mobi
Wed Feb 21 17:23:24 UTC 2018
Hello, Tim, thanks for trying it out again and thanks for your time
in debugging this issue and providing logs.
This problem is happening because we're not using fuego-host-scripts
in the derived fuego-release-test, what ends up resulting in the
jenkins' user not being properly mapped to the host's user.
That's my fault, I forgot that this would be required in the
derivate image as well.
I temporarily added that logic to build_and_run.sh (as a fixup commit)
so that you're able to test it out, please see the patch below:
diff --git a/build_and_run.sh b/build_and_run.sh
index e448ddf..0134759 100755
--- a/build_and_run.sh
+++ b/build_and_run.sh
@@ -50,6 +50,25 @@ fi
if [ "$1" = "up" ]; then
if [ -v clean_start ]; then
+ # FIXME: We should provide a simpler way of mapping jenkins uid
with
+ # hosts uid, that does not require the user (that's extending
+ # fuego-base) to have this logic in his docker build.
+ #
+ # Jenkins' uid/gid should probably be given as a docker-run
variable
+ # (as http_proxy/https_proxy, and set properly through fuego-base
+ # entrypoint.)
+ #
+ if [ "${UID}" == "0" ]; then
+ uid=$(id -u "${SUDO_USER}")
+ gid=$(id -g "${SUDO_USER}")
+ else
+ uid="${UID}"
+ gid=$(id -g "${USER}")
+ fi
+
+ sed -i "s/HOST_UID/${uid}/" ./fuego-ro/conf/fuego.conf
+ sed -i "s/HOST_GID/${gid}/" ./fuego-ro/conf/fuego.conf
+
docker build -t "${fuego_rt_image}" .
docker rm -f "${fuego_rt_container}"
docker run -dit --name ${fuego_rt_container} \
diff --git a/fuego-ro/conf/fuego.conf b/fuego-ro/conf/fuego.conf
index 0c95eae..96cdcc9 100644
--- a/fuego-ro/conf/fuego.conf
+++ b/fuego-ro/conf/fuego.conf
@@ -1,3 +1,3 @@
-docker_jenkins_uid=1000
-docker_jenkins_gid=987
+docker_jenkins_uid=HOST_UID
+docker_jenkins_gid=HOST_GID
As I mentioned in my FIXME message, its undesirable to require a derivate
image to contain specific build steps (as the one that's replacing the id
into fuego.conf),
so we might need to come up with a better approach to make this work.
The patch has already been pushed to
https://bitbucket.org/profusionmobi/fuego-release-test (master).
Can you please re-sync and test again?
Thank you
--
Guilherme
On Wed, Feb 21, 2018 at 11:48 AM, <Tim.Bird at sony.com> wrote:
> > -----Original Message-----
> > From: Guilherme Camargo
> > Tim, please sync the repo again (
> >
>
> https://bitbucket.org/profusionmobi/fuego-release-test
> > <https://bitbucket.org/profusionmobi/fuego-release-test> ).
> > I've force-pushed some changes.
> >
> > After that, you can just run `./build_and_run -c up`.
>
> My container doesn't start Jenkins. I'm not sure if this is
> a cascade error from some initial problem, but here is
> some information about my situation.
>
> I removed my repository, and did a new 'git clone' of the
> fuego-release-test repository. It doesn't appear to me to
> be updated recently:
>
> $ git branch -v
> * master 9d82c80 Initial commit
> $ git log
> commit 9d82c80d66a4de24ab97b38a16c8536849fea5bc
> Author: Guilherme Campos Camargo <guicc at profusion.mobi>
> Date: Tue Feb 6 02:13:20 2018 -0200
>
> Initial commit
>
> This repository has been created to store the resources needed for the
> generation of a Docker image, based on fuego-base, that has all the
> tools needed for the automation of the tests that are required for
> validating a new Fuego release with Fuego itself.
> ...
>
> I see that there are other branches with more recent commits (the
> 'ppa', and the 'fuego-base' branch. Should I be using one of these?
>
> Here's the error I saw while trying build_and_run.sh:
>
> -----
> Downloading/unpacking pexpect
> Downloading/unpacking ptyprocess>=0.5 (from pexpect)
> Downloading ptyprocess-0.5.2-py2.py3-none-any.whl
> Installing collected packages: pexpect, ptyprocess
> Successfully installed pexpect ptyprocess
> Cleaning up...
> ---> fd97e25fac02
> Removing intermediate container 5a3a3638cb21
> Step 4/7 : RUN echo "jenkins ALL = (root) NOPASSWD: /usr/bin/docker,
> /bin/sh" >> /etc/sudoers
> ---> Running in da0a1d824354
> ---> fb6468918ab5
> Removing intermediate container da0a1d824354
> Step 5/7 : COPY fuego-ro /fuego-ro
> ---> bbe089e90e5f
> Removing intermediate container 6a4b20b92a4e
> Step 6/7 : COPY tests /fuego-core/engine/tests
> ---> 57c7374dd097
> Removing intermediate container aa6b4c80255c
> Step 7/7 : COPY fuego-rt-entrypoint.sh /
> ---> 9ef063ab4a5b
> Removing intermediate container 32a90a4f3abb
> Successfully built 9ef063ab4a5b
> Successfully tagged fuego-rt:latest
> Error response from daemon: No such container: fuego-rt-container
> 6f513d49f117300672e1ba8782f0411448ffdf8e5258dd689797895cd79d8068
> Waiting for Jenkins on http://localhost:8090/fuego/...
> Creating job fuego-test.default.Functional.fuegotest
> fuego-rt-container
> [FAIL] Starting Jenkins Continuous Integration Server: jenkins failed!
> [ ok ] Starting network benchmark server.
> root at 6f513d49f117:/#
>
> -----
>
> And here is some investigation:
> -----
> root at 6f513d49f117:/# tail -n 20 /var/log/jenkins/jenkins.log
> INFO: Winstone shutdown successfully
> Feb 21, 2018 2:13:22 PM winstone.Logger logInternal
> SEVERE: Container startup failed
> java.io.FileNotFoundException: /var/cache/jenkins/war/META-INF/MANIFEST.MF
> (Permission denied)
> at java.io.FileOutputStream.open(Native Method)
> at java.io.FileOutputStream.<init>(FileOutputStream.java:221)
> at java.io.FileOutputStream.<init>(FileOutputStream.java:171)
> at winstone.HostConfiguration.getWebRoot(HostConfiguration.
> java:280)
> at winstone.HostConfiguration.<init>(HostConfiguration.java:83)
> at winstone.HostGroup.initHost(HostGroup.java:66)
> at winstone.HostGroup.<init>(HostGroup.java:45)
> at winstone.Launcher.<init>(Launcher.java:143)
> at winstone.Launcher.main(Launcher.java:352)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(
> NativeMethodAccessorImpl.java:57)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(
> DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at Main._main(Main.java:264)
> at Main.main(Main.java:112)
>
> root at 6f513d49f117:/# su - jenkins
> jenkins at 6f513d49f117:~$ cat /var/cache/jenkins/war/META-INF/MANIFEST.MF
> cat: /var/cache/jenkins/war/META-INF/MANIFEST.MF: Permission denied
> jenkins at 6f513d49f117:~$ ls -lad /var /var/cache /var/cache/jenkins
> /var/cache/jenkins/war /var/cache/jenkins/war/META-INF
> /var/cache/jenkins/war/META-INF/MANIFEST.MF
> ls: cannot access /var/cache/jenkins/war: Permission denied
> ls: cannot access /var/cache/jenkins/war/META-INF: Permission denied
> ls: cannot access /var/cache/jenkins/war/META-INF/MANIFEST.MF: Permission
> denied
> drwxr-xr-x 32 root root 4096 Feb 21 07:12 /var
> drwxr-xr-x 17 root root 4096 Feb 21 07:12 /var/cache
> drwxr-x--- 4 jenkins jenkins 4096 Feb 21 07:12 /var/cache/jenkins
> jenkins at 6f513d49f117:~$ exit
> logout
> root at 6f513d49f117:/# ls -lad /var /var/cache /var/cache/jenkins
> /var/cache/jenkins/war /var/cache/jenkins/war/META-INF
> /var/cache/jenkins/war/META-INF/MANIFEST.MF
> drwxr-xr-x 32 root root 4096 Feb 21 07:12 /var
> drwxr-xr-x 17 root root 4096 Feb 21 07:12 /var/cache
> drwxr-x--- 4 jenkins jenkins 4096 Feb 21 07:12 /var/cache/jenkins
> drwxr-xr-x 18 jenkins jenkins 4096 Feb 21 07:12 /var/cache/jenkins/war
> drwxr-xr-x 4 jenkins jenkins 4096 Feb 21 07:12
> /var/cache/jenkins/war/META-INF
> -rw-r--r-- 1 jenkins jenkins 129535 Feb 15 18:56
> /var/cache/jenkins/war/META-INF/MANIFEST.MF
> root at 6f513d49f117:/# su - jenkins
> jenkins at 6f513d49f117:~$ id
> uid=1000(jenkins) gid=987(jenkins) groups=987(jenkins)
> jenkins at 6f513d49f117:~$
> -----
> For reasons that escape me, it looks like these permissions are not
> allowing jenkins in the container to access the 'war' directory:
> drwxr-x--- 4 jenkins jenkins 4096 Feb 21 07:12 /var/cache/jenkins
>
> -----
> root at 6f513d49f117:/var/cache/jenkins# chmod o+rx .
> root at 6f513d49f117:/var/cache/jenkins# ls -la
> total 12
> drwxr-xr-x 4 jenkins jenkins 4096 Feb 21 07:12 .
> drwxr-xr-x 17 root root 4096 Feb 21 07:12 ..
> drwxr-xr-x 18 jenkins jenkins 4096 Feb 21 07:12 war
> root at 6f513d49f117:/var/cache/jenkins# su - jenkins
> jenkins at 6f513d49f117:~$ ls -lad /var /var/cache /var/cache/jenkins
> /var/cache/jenkins/war /var/cache/jenkins/war/META-INF
> /var/cache/jenkins/war/META-INF/MANIFEST.MF
> drwxr-xr-x 32 root root 4096 Feb 21 07:12 /var
> drwxr-xr-x 17 root root 4096 Feb 21 07:12 /var/cache
> drwxr-xr-x 4 jenkins jenkins 4096 Feb 21 07:12 /var/cache/jenkins
> drwxr-xr-x 18 jenkins jenkins 4096 Feb 21 07:12 /var/cache/jenkins/war
> drwxr-xr-x 4 jenkins jenkins 4096 Feb 21 07:12
> /var/cache/jenkins/war/META-INF
> -rw-r--r-- 1 jenkins jenkins 129535 Feb 15 18:56
> /var/cache/jenkins/war/META-INF/MANIFEST.MF
> -----
> However, maybe it's not that simple:
> -----
>
> jenkins at 6f513d49f117:~$ exit
> logout
> root at 6f513d49f117:/var/cache/jenkins# exit
> exit
> 1000072457 at tlinux:~/work/fuego/release-test-2018-02-19/fuego-release-test$
> ./build_and_run.sh up
> [FAIL] Starting Jenkins Continuous Integration Server: jenkins failed!
> [ ok ] Starting network benchmark server.
> root at 6f513d49f117:/# tail -n 20 /var/log/jenkins/jenkins.log
> INFO: Winstone shutdown successfully
> Feb 21, 2018 2:38:41 PM winstone.Logger logInternal
> SEVERE: Container startup failed
> java.io.FileNotFoundException: /var/cache/jenkins/war/META-INF/MANIFEST.MF
> (Permission denied)
> at java.io.FileOutputStream.open(Native Method)
> at java.io.FileOutputStream.<init>(FileOutputStream.java:221)
> at java.io.FileOutputStream.<init>(FileOutputStream.java:171)
> at winstone.HostConfiguration.getWebRoot(HostConfiguration.
> java:280)
> at winstone.HostConfiguration.<init>(HostConfiguration.java:83)
> at winstone.HostGroup.initHost(HostGroup.java:66)
> at winstone.HostGroup.<init>(HostGroup.java:45)
> at winstone.Launcher.<init>(Launcher.java:143)
> at winstone.Launcher.main(Launcher.java:352)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(
> NativeMethodAccessorImpl.java:57)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(
> DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at Main._main(Main.java:264)
> at Main.main(Main.java:112)
>
> root at 6f513d49f117:/# su - jenkins
> jenkins at 6f513d49f117:~$ ls -lad /var /var/cache /var/cache/jenkins
> /var/cache/jenkins/war /var/cache/jenkins/war/META-INF
> /var/cache/jenkins/war/META-INF/MANIFEST.MF
> ls: cannot access /var/cache/jenkins/war: Permission denied
> ls: cannot access /var/cache/jenkins/war/META-INF: Permission denied
> ls: cannot access /var/cache/jenkins/war/META-INF/MANIFEST.MF: Permission
> denied
> drwxr-xr-x 32 root root 4096 Feb 21 07:12 /var
> drwxr-xr-x 17 root root 4096 Feb 21 07:12 /var/cache
> drwxr-xr-x 4 jenkins jenkins 4096 Feb 21 07:12 /var/cache/jenkins
>
> I'm running this on Ubuntu 14.04, with docker 17.05.0-ce
>
> -----
> OK - in summary:
>
> - I think I might be running the wrong branch of fuego-release-test
> - I have weird permission problems in my docker container, for the fuego
> user and group
> that are preventing jenkins from starting
> - the jenkins user id and group id don't match my local user's user id and
> group id
>
> Any help would be appreciated.
> -- Tim
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxfoundation.org/pipermail/fuego/attachments/20180221/45b972e7/attachment-0001.html>
More information about the Fuego
mailing list