[Fuego] benchmark metrics changes (was RE: [PATCH] tests.info: reboot has only a time plot at the moment)

[Daniel Sangorrin]

Hi Tim,

> -----Original Message-----
> From: Bird, Timothy [mailto:Tim.Bird at am.sony.com]
> Sent: Friday, November 18, 2016 3:17 AM
> OK - that's interesting.   Putting extra information in the Jenkins interface
> would be really useful.  I'd like to figure out how to get direct links to the
> test log file for a particular test run.  The logs are sitting there under
> /var/lib/jenkins/userContent/fuego.logs, which is visible via the web
> server interface.  But it would be convenient to have a direct link.

You can add a "Set build description" section and use these lines:
Dbench benchmark<p><b><a href="/userContent/jta.logs/${JOB_NAME}/testlogs/${NODE_NAME}.${BUILD_ID}.${BUILD_NUMBER}.log">Test Result</a></b></p>

[Note] substitute testlogs by devlogs for the complete logs
[Note] This works on AGL-JTA, I haven't tested it in Fuego yet because I'm figuring 
out how to upgrade it in a clean and maintainable way.
 
> However, with regard to the JTA-CIAT solution, I'm not an XML fan.
> I'm reminded me of an old saying:
> "Sometimes, when a developer is faced with a technical problem, they
> decide to solve it using XML.  Now they have 2 problems."  :-)

Haha well, that's a question of taste. I find python's library xmltodict (same one
as the one AGL-JTA is using) quite easy to use, and the XML format is easy to read.
The real power of XML comes from its ability to validate against an XML Schema.
Another option is to use JSON, which nowadays  has very similar functionality.

I wouldn't mind so much about the format as long as there are good mature tools for 
parsing and visualizing the test results.

> I've been toying with converting some results to markdown, for
> another front end I've been experimenting with.
> This might work for reports as well, if we can settle on something
> that works for both online (web) and printed (pdf) docs.

Sounds good. 

Once we have a single common output format (e.g. XML or JSON) we can
make several visualization tools. 

> Another thing would be to color-code the logs, with some post-processing.
> Scanning visually through the Jenkins console logs for the Fuego error
> is time consuming. This could be helped by a post-processor that just
> added '<font color="red">' and '</font>' around some Fuego messages.

That would be great.

Best regards,
Daniel