[Fuego] release test issues

Guilherme Camargo guicc at profusion.mobi
Fri Feb 23 16:23:20 UTC 2018


Hello, Tim

I think we're getting closer to the root cause.

It's still not very clear why, but there seems to be some sort of race
condition when, while running the fuego-release-test as a daemon we try to
execute fuego-rt-entrypoint.sh. For some reason, we might be trying to run
the
`ftc` commands while the uid/gid of jenkins have not been mapped yet in the
original entrypoint.sh.

With my newest change, we're calling `docker run` only once, guaranteeing
that:
 - fuego-base "entrypoint.sh"
 - fuego-release-test "fuego-rt-entrypoint.sh"
 - and "/bin/bash"
are executed sequentially.

Would you please try again with the new commits that have been pushed to
master?

Steps are:

```
git clone https://bitbucket.org/profusionmobi/fuego-release-test
cd fuego-release-test
./build_and_run -c up

Then try to access fuego in localhost:8080/fuego/

You can also pass -p PORT to build_and_run and access fuego from that
location instead.

```

Thank you very much

--
Guilherme


On Thu, Feb 22, 2018 at 6:31 PM, <Tim.Bird at sony.com> wrote:

> But, just because Jenkins started, doesn't mean everything is working.
> I got a Jenkins "Oops" when trying to build the
> fuego-test.default.Functional.fuegotest job.
>
> Here's the first part of the console log:
> -----
> A problem occurred while processing the request. Please check our bug
> tracker to see if a similar problem has already been reported. If it is
> already reported, please vote and put a comment on it to let us gauge the
> impact of the problem. If you think this is a new issue, please file a new
> issue. When you file an issue, make sure to add the entire stack trace,
> along with the version of Jenkins and relevant plugins. The users list
> might be also useful in understanding what has happened.
>
> Stack trace
>
> javax.servlet.ServletException: org.apache.commons.jelly.JellyTagException:
> jar:file:/var/cache/jenkins/war/WEB-INF/lib/jenkins-core-2.3
> 2.1.jar!/hudson/model/Run/console.jelly:65:27: <j:whitespace> Failed to
> load the key: hudson.console.AnnotatedLargeText.consoleAnnotator
>         at org.kohsuke.stapler.jelly.JellyFacet$1.dispatch(JellyFacet.j
> ava:103)
>         at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:746)
>         at org.kohsuke.stapler.Stapler.invoke(Stapler.java:876)
>         at org.kohsuke.stapler.MetaClass$10.dispatch(MetaClass.java:362)
>         at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:746)
>         at org.kohsuke.stapler.Stapler.invoke(Stapler.java:876)
>         at org.kohsuke.stapler.MetaClass$5.doDispatch(MetaClass.java:236)
>         at
> > -----Original Message-----
> > From: Tim Bird
> > > -----Original Message-----
> > > From: Guilherme Camargo
> > >
> > > I made a few more modifications in the way that the uid/gid are set on
> > Fuego
> > >
> > (https://bitbucket.org/profusionmobi/fuego/commits/fe24e93ce191c461f4e
> > > fa55aaa345a319b1b8fd6)
> > > and also made the needed changes on fuego-release-test to match those.
> > > Hopefully they're more robust now. I plan to send all the new changes
> by
> > e-
> > > mail
> > > to the list, but I just wanted to check if you're able to run it
> locally, so
> > > that you can get a feeling about this, before we move on.
> > >
> > >
> > > Can you please try again with:
> > >
> > >
> > > ```
> > > git clone https://bitbucket.org/profusionmobi/fuego-release-test
> > > cd fuego-release-test
> > > ./build_and_run -c up
> >
> > Still no joy.
> >
> > Step 7/7 : COPY fuego-rt-entrypoint.sh /
> >  ---> 4e39b71fb0d5
> > Removing intermediate container a5e9b9e4244d
> > Successfully built 4e39b71fb0d5
> > Successfully tagged fuego-rt:latest
> > fuego-rt-container
> > a8750b71dc87bc1ad61ed26954c9484e79376d019fbd563abf86e1fcec86c70b
> > Waiting for Jenkins...
> > Creating job fuego-test.default.Functional.fuegotest
> > fuego-rt-container
> > [FAIL] Starting Jenkins Continuous Integration Server: jenkins failed!
> > [ ok ] Starting network benchmark server.
> > root at a8750b71dc87:/# tail -n 20 /var/log/jenkins/jenkins.log
> > INFO: Winstone shutdown successfully
> > Feb 22, 2018 9:15:14 PM winstone.Logger logInternal
> > SEVERE: Container startup failed
> > java.io.FileNotFoundException: /var/cache/jenkins/war/META-
> > INF/MANIFEST.MF (Permission denied)
> >       at java.io.FileOutputStream.open(Native Method)
> >       at java.io.FileOutputStream.<init>(FileOutputStream.java:221)
> >       at java.io.FileOutputStream.<init>(FileOutputStream.java:171)
> >       at
> > winstone.HostConfiguration.getWebRoot(HostConfiguration.java:280)
> >       at winstone.HostConfiguration.<init>(HostConfiguration.java:83)
> >       at winstone.HostGroup.initHost(HostGroup.java:66)
> >       at winstone.HostGroup.<init>(HostGroup.java:45)
> >       at winstone.Launcher.<init>(Launcher.java:143)
> >       at winstone.Launcher.main(Launcher.java:352)
> >       at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> >       at
> > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.j
> > ava:57)
> >       at
> > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAcces
> > sorImpl.java:43)
> >       at java.lang.reflect.Method.invoke(Method.java:606)
> >       at Main._main(Main.java:264)
> >       at Main.main(Main.java:112)
> >
> > ----
> > But check out this sequence of actions:
> >
> > -----
> > root at a8750b71dc87:/# id jenkins
> > uid=2060932290(jenkins) gid=2060452353(jenkins)
> > groups=2060452353(jenkins)
> > ...
> > root at a8750b71dc87:/# ls -ld /var /var/cache /var/cache/jenkins
> > /var/cache/jenkins/war /var/cache/jenkins/war/META-INF
> > /var/cache/jenkins/war/META-INF/MANIFEST.MF
> > drwxr-xr-x 32 root    root      4096 Feb 22 21:14 /var
> > drwxr-xr-x 17 root    root      4096 Feb 22 21:14 /var/cache
> > drwxr-x---  4 jenkins jenkins   4096 Feb 22 21:14 /var/cache/jenkins
> > drwxr-xr-x 18 jenkins jenkins   4096 Feb 22 21:14 /var/cache/jenkins/war
> > drwxr-xr-x  4 jenkins jenkins   4096 Feb 22 21:14
> > /var/cache/jenkins/war/META-INF
> > -rw-r--r--  1 jenkins jenkins 129535 Feb 22 18:17
> > /var/cache/jenkins/war/META-INF/MANIFEST.MF
> > root at a8750b71dc87:/# su - jenkins
> > jenkins at a8750b71dc87:~$ ls -ld /var /var/cache /var/cache/jenkins
> > /var/cache/jenkins/war /var/cache/jenkins/war/META-INF
> > /var/cache/jenkins/war/META-INF/MANIFEST.MF
> > drwxr-xr-x 32 root    root      4096 Feb 22 21:14 /var
> > drwxr-xr-x 17 root    root      4096 Feb 22 21:14 /var/cache
> > drwxr-x---  4 jenkins jenkins   4096 Feb 22 21:14 /var/cache/jenkins
> > drwxr-xr-x 18 jenkins jenkins   4096 Feb 22 21:14 /var/cache/jenkins/war
> > drwxr-xr-x  4 jenkins jenkins   4096 Feb 22 21:14
> > /var/cache/jenkins/war/META-INF
> > -rw-r--r--  1 jenkins jenkins 129535 Feb 22 18:17
> > /var/cache/jenkins/war/META-INF/MANIFEST.MF
> > jenkins at a8750b71dc87:~$ cat /var/cache/jenkins/war/META-
> > INF/MANIFEST.MF
> > Manifest-Version: 1.0
> > Jenkins-Version: 2.32.1
> > Implementation-Version: 2.32.1
> > Hudson-Version: 1.395
> > Build-Jdk: 1.7.0_21
> > Built-By: kohsuke
> > Created-By: Apache Maven 3.3.9
> > Main-Class: Main
> > Archiver-Version: Plexus Archiver
> >
> > Name: images/48x48/red_anime.gif
> > SHA-256-Digest: 2ktTeqkcRPD3cWv0z9xPu6Hf3kMNxoh55qzYKVMdCwk=
> > ... [lots of lines omitted]
> >
> > Name: scripts/yui/assets/skins/sam/button.css
> > SHA-256-Digest: Yp390BkBCcwkhhCQ4Fqj06dTN5qg/zI1QbaR3Gkl6LY=
> >
> > jenkins at a8750b71dc87:~$ exit
> > logout
> > root at a8750b71dc87:/# service jenkins start
> > [ ok ] Starting Jenkins Continuous Integration Server: jenkins.
> > root at a8750b71dc87:/#
> > -----
> > If the jenkins log message is correct, there's something really
> > weird going on here.  The permissions look OK, and the 'jenkins'
> > user can access the MANIFEST.MF file, and a subsequent
> > attempt to start jenkins (as root) works.  So I'm not sure what's going
> on.
> >  -- Tim
> >
> > _______________________________________________
> > Fuego mailing list
> > Fuego at lists.linuxfoundation.org
> > https://lists.linuxfoundation.org/mailman/listinfo/fuego
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxfoundation.org/pipermail/fuego/attachments/20180223/360ec045/attachment-0001.html>


More information about the Fuego mailing list