[Fuego] benchmark metrics changes (was RE: [PATCH] tests.info: reboot has only a time plot at the moment)

Bird, Timothy Tim.Bird at am.sony.com
Thu Nov 17 18:16:50 UTC 2016

> -----Original Message-----
> From: Daniel Sangorrin on  Thursday, November 17, 2016 12:56 AM> 
> > -----Original Message-----
> > From: Bird, Timothy [mailto:Tim.Bird at am.sony.com]
> > I'm not sure why there are two different plotting systems, but the
> matplotlib
> > (plot.png) one doesn't seem configured right.  It only shows data for one
> run,
> > that I can see, and it's zoom factor is way off.
> Actually, if you try the "reboot" test with my patch, the plot.png
> should be working. Please let me know otherwise.
> Dhrystone was also working if I remember correctly.

I'll double-check this.  I keep getting my plot data messed up.
I believe it's because the test numbering re-sets every time
I build a new docker container.

I'm considering either putting some of the info from /home/jenkins/fuego/jobs
into /userdata as well, or adding a manual tool to sync nextBuildNumber
when a new container is deployed.

> From a first glance at the AGL-JTA-core repository, they have a "common"
> folder
> that includes almost the same tests as Fuego (some tests are new and some
> are missing).
> Most of them have unmodified parser.py files (apart from the JTA-Fuego
> renaming).
> However, I observe that each test also has a file "create_xml_testname.py".
> From a quick
> read it looks like they are parsing the logs into xml files, and at the same time
> saving information
> such as timing or build number.  I haven't checked yet but my guess is they
> are later
> converting the XML files into HTML so they can be read from the Jenkins
> interface.

OK - that's interesting.   Putting extra information in the Jenkins interface
would be really useful.  I'd like to figure out how to get direct links to the
test log file for a particular test run.  The logs are sitting there under
/var/lib/jenkins/userContent/fuego.logs, which is visible via the web
server interface.  But it would be convenient to have a direct link.

However, with regard to the JTA-CIAT solution, I'm not an XML fan. 
I'm reminded me of an old saying:
"Sometimes, when a developer is faced with a technical problem, they
decide to solve it using XML.  Now they have 2 problems."  :-)

> # I will check what the XML format looks like next week and report


I've been toying with converting some results to markdown, for
another front end I've been experimenting with.
This might work for reports as well, if we can settle on something
that works for both online (web) and printed (pdf) docs.

Another thing would be to color-code the logs, with some post-processing.
Scanning visually through the Jenkins console logs for the Fuego error
is time consuming. This could be helped by a post-processor that just
added '<font color="red">' and '</font>' around some Fuego messages.

 -- Tim

More information about the Fuego mailing list