[Fuego] Fuego next status update

[Tim.Bird at sony.com]

> -----Original Message-----
> From: daniel.sangorrin at toshiba.co.jp
> > From: Tim.Bird at sony.com
> >
> > Hello Fuego users and developers,
> >
> > Here is a quick status update for Fuego.  Lately I've been working on
> changes to the batch
> > processing system.  The new batch_id code seems to work pretty well.  It's
> in the 'next' branch,
> > and it needs more testing to figure out the best way to pass the batch_id
> from parent test
> > jobs to child test jobs.  But querying runs using batch_id works fine, and
> this is leading
> > to an overhaul of the batch system (moving from the testplan json files to
> Fuego tests
> > that are themselves collections of sub-tests).  In the process, this will lead
> to better
> > facilities to compose complicated test pipelines natively in Fuego (without
> resorting to
> > Jenkins pipelines).
> 
> Sounds good. It would be nice to run various tests at the same time in
> parallel.
> 
> > Also, I've worked on integrating the 'fserver' code, which is the Fuego
> > server code for managing job requests between Fuego sites.  This code
> may end up
> > supporting other functionality in the future, such as a binary test package
> cache, and sharing
> > tests outside of the fuego-core repository.  It's still in the prototype stage,
> but having it
> > included by default in the container will allow people to look at it and
> experiment with it
> > if they'd like.
> 
> I saw the code in ftc, but are you also going to include fserver in the
> Dockerfile or a install script?

Yes.  fserver is now installed in the docker container, but it is not started
by default.  I need to write up some instructions for running it.  There's
a script that will start up a python-based web server, if people want to try it
out.  However, fserver is a CGI script that is intended to be used with
a real web server if you want decent performance and scalability.

...
> > I've done a bit of refactoring of the directory structure for Fuego.  I've
> removed the 'engine'
> > directory in fuego-core, and I've placed fuego-core inside the fuego
> directory (parallel
> > to fuego-ro and fuego-rw).  This puts the fuego-core git repository inside
> the fuego git repository,
> > which might be a bit confusing.  However, since this configuration matches
> the directory
> > structure inside the docker container, it allowed me to remove an ugly hack
> in ftc that dealt
> > with this difference.  I also think it's easier for users conceptually.
> By the way, for those who want to see the git log of a file, add the option --
> follow to see the whole history of commits.
> Eg: git log --follow scripts/parser/common.py

Thanks for the tip.  This will be helpful.

> > The downside of this directory refactoring is that patches made against the
> 'master' branch
> > will not apply cleanly in the 'next' branch.  Tests that have 'engine' in their
> paths should
> > still work, as I've left a compatibility symlink in the repository.  However,
> managing patches
> > will be difficult.  For this reason, I'd like to accelerate the 1.5 release, and try
> to do it much
> > faster than the 1.4 release.  I'm actually hoping to get it done by the
> beginning of March.
> > It will be a much shorter release (only 2 months or so).  I want to shorten
> the release
> > window in order to reduce the amount of work spent on patch fixups that
> are required due
> > to the directory refactoring.
> >
> > Finally - I'm moving the main fuego repositories from my personal
> bitbucket account (tbird20d)
> > to a 'fuegotest' account.  The new git url will be:
> https://bitbucket.org/fuegotest/fuego.git
> > A change to the install process now only requires that you download the
> 'fuego' repository.
> > During install, fuego will download 'fuego-core' for you, so this shortens the
> installation
> > instructions a bit.
> 
> Will you still use the personal ones for development?

I don't intend to.  If I have work in progress, I'll put that on branches
in the fuegotest repositories.

> 
> > This will all get written up for the 1.5 release notes.  The main reason for
> this status update
> > is to let you all know about the shortened release.  If you have a feature
> pending that you'd like
> > to get in to this release, please let me know.  And of course, if you are
> interested in these
> > features, please try out the 'next' branch (which is now maintained in the
> fuegotest bitbucket
> > account) and let me know what you think.
> 
> I see you are adding a lot of changes in the background ;)

I'm trying to clean up a few loose ends. :-)
 -- Tim