[Fuego] RFC: Fuego's version up and other changes

Daniel Sangorrin daniel.sangorrin at toshiba.co.jp
Thu Jan 12 09:18:55 UTC 2017


> From: Ibe.Kengo at bx.MitsubishiElectric.co.jp [mailto:Ibe.Kengo at bx.MitsubishiElectric.co.jp]
> Hi, Daniel-san,
> 
> Thank you for prompt help.
> I was able to run tests in my Docker!! :)
> 
> I'm going to try to use some function of your Fuego.
> If I have some question, let me ask you some question.

Ok, that's great!
I'll be glad to answer your questions or ideas for improving it.

Thanks,
Daniel

> ----- Original Message -----
> 件名: RE: [Fuego] RFC: Fuego's version up and other changes
> 送信者: "Daniel Sangorrin" <daniel.sangorrin at toshiba.co.jp>
> 送信日時: 2017年01月12日(木) 17:45:20
> 宛先: Ibe.Kengo at bx.MitsubishiElectric.co.jp
> Cc: fuego at lists.linuxfoundation.org
> 
> > Hi Kengo-san,
> >
> > I'm sorry, it seems that fuego-rw didn't get pushed. The reason is
> because it's included in .gitignore!!
> > I have pushed it now. Please remove your current fuego-rw and do a git
> pull. It should work properly this time.
> >
> > Thanks
> > Daniel
> >
> >
> > > -----Original Message-----
> > > From: Ibe.Kengo at bx.MitsubishiElectric.co.jp [mailto:Ibe.Kengo at bx.
> MitsubishiElectric.co.jp]
> > > Sent: Thursday, January 12, 2017 5:16 PM
> > > To: Daniel Sangorrin
> > > Cc: fuego at lists.linuxfoundation.org
> > > Subject: Re: [Fuego] RFC: Fuego's version up and other changes
> > >
> > > Hi, Daniel-san
> > >
> > > Thank you for your prompt reply.
> > > I have more a few question.
> > >
> > > > Humm You should not need to change any permissions.
> > > OK.
> > > I removed the Fuego and the Fuego-core,
> > > after that, I redownloaded the Fuego and the Fuego-core and
> rebuilded
> > > and start Docker.
> > > I can see following permissions of directories. Is it correct?
> > > ==
> > > melco at debian8:~/$ ./fuego/fuego-host-scripts/docker-start-container.
> sh
> > > Starting Fuego container (fuego-container)
> > > [ ok ] Starting Jenkins Continuous Integration Server: jenkins.
> > >
> > > root at debian8:/# ls -al
> > > total 84
> > > drwxr-xr-x  65 root    root    4096 Jan 12 07:26 .
> > > drwxr-xr-x  65 root    root    4096 Jan 12 07:26 ..
> > > -rwxr-xr-x   1 root    root       0 Jan 12 07:26 .dockerenv
> > > drwxr-xr-x   2 root    root    4096 Jan 12 00:53 bin
> > > drwxr-xr-x   2 root    root    4096 Sep 12 04:09 boot
> > > drwxr-xr-x   5 root    root     380 Jan 12 07:26 dev
> > > drwxr-xr-x  96 root    root    4096 Jan 12 07:26 etc
> > > drwxr-xr-x   4 jenkins jenkins 4096 Jan 12 07:07 fuego-core
> > > drwxr-xr-x   5 jenkins jenkins 4096 Jan 12 07:03 fuego-ro
> > > drwxr-xr-x   2 root    root    4096 Jan 12 07:26 fuego-rw
> > > drwxr-xr-x   2 root    root    4096 Sep 12 04:09 home
> > > ~
> > >
> > > melco at debian8:~/$ ls -al fuego
> > > 合計 68
> > > drwxr-xr-x 8 melco melco 4096  1月 12 16:26 .
> > > drwxr-xr-x 4 melco melco 4096  1月 12 16:02 ..
> > > -rw-r--r-- 1 melco melco   42  1月 12 16:03 .dockerignore
> > > drwxr-xr-x 8 melco melco 4096  1月 12 16:03 .git
> > > -rw-r--r-- 1 melco melco    9  1月 12 16:03 .gitignore
> > > -rw-r--r-- 1 melco melco 1482  1月 12 16:03 CHANGELOG
> > > -rw-r--r-- 1 melco melco 4810  1月 12 16:03 Dockerfile
> > > -rw-r--r-- 1 melco melco 1492  1月 12 16:03 LICENSE
> > > -rw-r--r-- 1 melco melco 3414  1月 12 16:03 README
> > > -rw-r--r-- 1 melco melco   85  1月 12 16:03 VERSION
> > > drwxr-xr-x 3 melco melco 4096  1月 12 16:03 docs
> > > drwxr-xr-x 3 melco melco 4096  1月 12 16:03 frontend-install
> > > drwxr-xr-x 2 melco melco 4096  1月 12 16:03 fuego-host-scripts
> > > drwxr-xr-x 5 melco melco 4096  1月 12 16:03 fuego-ro
> > > drwxr-xr-x 2 root  root  4096  1月 12 16:26 fuego-rw
> > > -rwxr-xr-x 1 melco melco   48  1月 12 16:03 install.sh
> > >
> > > ==
> > > “melco” user is only user on host PC which is Debian8.
> > > I’m concerned about fuego-rw permissions and owner.
> > >
> > > > The script is supposed to use your user's uid/gid for the jenkins
> > > daemon.
> > > Do you mean that I need to make “Jenkins” user on host PC?
> > > Or
> > > Can I use “melco” user inside Docker?
> > >
> > > > However, I have only tested it on my pc which has a single user.
> > > > Do you have more than one user?
> > > I have one user. I'v not made "jenkins" user on host PC.
> > >
> > > I’ll check other point following your advice.
> > > Thank you so much.
> > > Kengo
> > >
> > >
> > > ----- Original Message -----
> > > 件名: RE: [Fuego] RFC: Fuego's version up and other changes
> > > 送信者: "Daniel Sangorrin" <daniel.sangorrin at toshiba.co.jp>
> > > 送信日時: 2017年01月12日(木) 15:44:34
> > > 宛先: Ibe.Kengo at bx.MitsubishiElectric.co.jp
> > > Cc: fuego at lists.linuxfoundation.org
> > >
> > > > Hi Kengo-san,
> > > >
> > > > Thanks a lot for your feedback.
> > > >
> > > > > -----Original Message-----
> > > > > From: Ibe.Kengo at bx.MitsubishiElectric.co.jp [mailto:Ibe.Kengo at bx.
> > > MitsubishiElectric.co.jp]
> > > > > Sent: Thursday, January 12, 2017 2:52 PM
> > > > > To: Daniel Sangorrin
> > > > > Cc: fuego at lists.linuxfoundation.org
> > > > > Subject: Re: [Fuego] RFC: Fuego's version up and other changes
> > > > >
> > > > > Hi, Daniel-san.
> > > > >
> > > > > I’m very interested in your work!
> > > > > So I tried to run your build scripts to create Docker image
> today,
> > > > > then I could run new Fuego.
> > > > > And, I added Docker node and test cases to use following command.
> > > > > ==
> > > > >   $ git clone --branch next https://bitbucket.org/nirrognas/fuego.git
> 
> > >
> > > > >   $ git clone --branch next https://bitbucket.org/nirrognas/fuego-core.git
> 
> > >
> > > > >
> > > > >   $ cd fuego/
> > > > >   $ ./install.sh
> > > > >   $ ./fuego-host-scripts/docker-create-container.sh
> > > > >   $ ./fuego-host-scripts/docker-start-container.sh
> > > > >   docker# fuego-create-node --board docker
> > > > >   docker# fuego-create-jobs --board docker --testplan testplan_
> > > docker
> > > > > --distrib nosyslogd.dist
> > > > > ==
> > > > >
> > > > > I can see  new node and new test jobs.
> > > > > But, I can’t run tests like Dhrystone on Docker.
> > > > >
> > > > > So far, I had one small problem that is “Failed to mkdirs: /
> fuego-rw
> > > /
> > > > > buildzone”
> > > > > This reason is the permission or owner of “fuego-rw”,
> > > > > so I added the write permission to “fuego-rw” diredtory.
> > > >
> > > > Humm You should not need to change any permissions.
> > > > The script is supposed to use your user's uid/gid for the jenkins
> > > daemon.
> > > > However, I have only tested it on my pc which has a single user.
> > > > Do you have more than one user? Could you check that the jenkins
> uid/
> > > gid
> > > > is the same as yours? (you can check on /etc/passwd inside and
> outside
> > > docker)
> > > > Please also check that the fuego-rw's owner is your user (e.g.:
> don't
> > > git clone with
> > > > a different user or root)
> > > >
> > > > > I have solved this problem.
> > > > >
> > > > > After that, I got the error message like following in console
> log.
> > > > > ====
> > > > > Started by user anonymous
> > > > > Building remotely on docker in workspace /fuego-rw/buildzone
> > > > > [buildzone] $ /bin/sh -xe /tmp/hudson1255554151521556820.sh
> > > > > + export Reboot=false
> > > > > + Reboot=false
> > > > > + export Rebuild=true
> > > > > + Rebuild=true
> > > > > + export Target_Cleanup=true
> > > > > + Target_Cleanup=true
> > > > > + export TESTDIR=Benchmark.Dhrystone
> > > > > + TESTDIR=Benchmark.Dhrystone
> > > > > + export TESTNAME=Dhrystone
> > > > > + TESTNAME=Dhrystone
> > > > > + export DISTRIB=nosyslogd.dist
> > > > > + DISTRIB=nosyslogd.dist
> > > > > + export TESTPLAN=testplan_docker
> > > > > + TESTPLAN=testplan_docker
> > > > > + timeout --signal=9 3m /bin/bash
> > > > > /fuego-core/engine/tests/Benchmark.Dhrystone/Dhrystone.sh
> > > > > board overlay: boards/docker.board
> > > > > using boards/docker.board board overlay
> > > > > using nosyslogd.dist overlay
> > > > > BATCH_TESTPLAN is not set, using testplan_docker.json testplan
> > > > > Traceback (most recent call last):
> > > > >   File "/fuego-core/engine/scripts/ovgen/ovgen.py", line 615, in
> <
> > > module
> > > > > >
> > > > >     run()
> > > > >   File "/fuego-core/engine/scripts/ovgen/ovgen.py", line 607, in
> run
> > > > >     generateProlog(output, ofcls, classes, testPlans, testSpecs)
> > > > >   File "/fuego-core/engine/scripts/ovgen/ovgen.py", line 375, in
> > > > > generateProlog
> > > > >     outfile = open(outFilePath, "w")
> > > > > IOError: [Errno 2] No such file or directory: '/fuego-rw/work/
> docker
> > > _
> > > > > prolog.sh'
> > > > >
> > > > > *** ABORTED ***
> > > > >
> > > > > Fuego error reason: Error while prolog.sh file generation
> > > > >
> > > > > Build step 'Execute shell' marked build as failure
> > > > > [description-setter] Description set:
> > > > > Finished: FAILURE
> > > > > ====
> > > > >
> > > > > Did I miss to run test cases on Docker node?
> > > > > If you know this solution, could you give me some advice?
> > > >
> > > > Could you check that fuego-rw is correctly mounted inside docker?
> > > > outside-docker$ touch fuego-rw/hoge
> > > > inside-docker$ ls /fuego-rw
> > > > hoge <-- does it appear?
> > > >
> > > > On my PC (ubuntu 16.04, docker version 1.12.3 ) it's always
> succesful
> > > so I will
> > > > try on a different one. Which OS and docker version are you using?
> > > >
> > > > Thanks,
> > > > Daniel
> > > >
> > > > > Thank you.
> > > > >
> > > > > Best Regard.
> > > > > Kengo Ibe
> > > > >
> > > > > ----- Original Message -----
> > > > > 件名: [Fuego] RFC: Fuego's version up and other changes
> > > > > 送信者: "Daniel Sangorrin" <daniel.sangorrin at toshiba.co.jp>
> > > > > 送信日時: 2017年01月11日(水) 09:28:07
> > > > > 宛先: "Bird, Timothy" <Tim.Bird at am.sony.com>, fuego at lists.
> > > > > linuxfoundation.org
> > > > >
> > > > > > Hi everyone,
> > > > > >
> > > > > > I've been working on Fuego's upgrade for a few weeks, and in
> the
> > > > > process I
> > > > > > have made quite a few changes.
> > > > > > You can check it by following these instructions. I'd really
> > > > > appreciate
> > > > > > your feedback:
> > > > > >
> > > > > > Quickstart:
> > > > > >   $ git clone --branch next https://bitbucket.org/nirrognas/fuego.git
> 
> > >
> > > > > >   $ git clone --branch next https://bitbucket.org/nirrognas/fuego-core.git
> 
> > >
> > > > >
> > > > > >   $ cd fuego/
> > > > > >   $ ./install.sh
> > > > > >   $ ./fuego-host-scripts/docker-create-container.sh
> > > > > >   $ ./fuego-host-scripts/docker-start-container.sh
> > > > > >   docker# fuego-create-node --board docker
> > > > > >   docker# fuego-create-jobs --board docker --testplan testplan
> _
> > > docker
> > > > > > --distrib nosyslogd.dist
> > > > > >   Then, start the "docker.testplan_docker.batch" job which
> will
> > > > > trigger
> > > > > > the rest of jobs.
> > > > > >
> > > > > > Here is a list of the most fundamental modifications I've
> added:
> > > > > > - I automatized the version upgrade process. So far I have
> updated
> > > > > Jenkins
> > > > > > 3 times quite smoothly.
> > > > > >    The current version is the latest LTS version (2.31.1 LTS)
> > > > > > - I reduced plugin dependencies to the minimum (only 2, and
> they
> > > can
> > > > > be
> > > > > > made optional).
> > > > > > - I removed dependencies on all groovy scripts.
> > > > > > - Userdata is gone. Instead we now have fuego-ro and fuego-rw,
> > > which
> > > > > > together with fuego-core
> > > > > >    are mounted as external docker volumes. Only fuego-rw,
> which
> > > > > contains
> > > > > > logs and build folders,
> > > > > >    is mounted in RW mode. I also fixed the uid/gid of jenkins
> so
> > > that
> > > > > it
> > > > > > matches the one in the host.
> > > > > >    This means you can develop comfortably on your host, with
> GUI
> > > tools,
> > > > > > and don't fear a buggy
> > > > > >    script deleting your folders.
> > > > > > - I added a  tool for generating jobs from testplans.
> Creating
> > > jobs
> > > > > from
> > > > > > testplans is powerful.
> > > > > >    It allows you, for example, to easily specify the timeout
> for
> > > each
> > > > > test
> > > > > > on your board.
> > > > > >    The tool also generates the corresponding batch job. For
> > > example,
> > > > > you
> > > > > > can easily trigger all
> > > > > >    tests for your board and a specific testplan periodically.
> > > > > >    [Note] This will also enable the creation of scripts for
> > > comparing
> > > > > test
> > > > > > results across
> > > > > >    different boards and test parameters (on my TODO list).
> > > > > >
> > > > > > Other less fundamental changes include:
> > > > > > - I added support for using the docker container itself as a
> > > target
> > > > > board
> > > > > > (e.g.: for a quickstart or unit tests)
> > > > > > - I fixed the flot plugin (java and javascript fixes)
> > > > > > - Now you can click on a node and see which jobs (and
> testplans)
> > > are
> > > > > > assigned to your board. This was
> > > > > >   one of my old feature requests and makes Fuego's GUI easier
> to
> > > use.
> > > > > > - I added quite a few fixes and improvements to the core
> engine
> > > > > scripts
> > > > > > and tests (too detailed to describe here).
> > > > > > - I added Excel output support for IOzone
> > > > > > - Faster docker build time (ARMhf compiler installation is now
> > > > > optional,
> > > > > > no latex..)
> > > > > > - I put a fixed name to the fuego container, instead of the
> > > > > > "last_container_id" file.
> > > > > > - I removed the inotify script
> > > > > > - I added scripts for easily removing the docker container/
> image.
> > > > > >
> > > > > > I have to split the current "huge" patch into smaller chunks
> and
> > > test
> > > > > it
> > > > > > further but it would
> > > > > > be nice to get your feedback about the approach in general.
> > > > > >
> > > > > > Tim: I haven't updated ftc nor your "Christmas commits" yet.
> > > > > >
> > > > > > Best regards
> > > > > > Daniel
> > > > > >
> > > > > >
> > > > > > _______________________________________________
> > > > > > Fuego mailing list
> > > > > > Fuego at lists.linuxfoundation.org
> > > > > > https://lists.linuxfoundation.org/mailman/listinfo/fuego
> > > > > >
> > > > >
> > > >
> > > >
> > > >
> > > >
> > >
> >
> >
> >
> >
> 





More information about the Fuego mailing list