[Fuego] Discussion about Fuego unified results format

[Kevin Hilman]

"Bird, Timothy" <Tim.Bird at sony.com> writes:

> Based on discussion in our call yesterday, I have a few notes I'd like to make.
> First - for those not involved in the call, here is some introduction.  We are
> working, for the 1.2 release, on unifying the results format produced by Fuego
> tests, to make report generation easier, and support (ultimately) multiple
> report output formats (e.g. HTML, XML, PDF, Excel).  The intent is to support
> all current AGL-JTA reports, previous Fuego reports, and new ones envisioned
> for the system (in the 1.2 release).
>
> Note from the call and about this work are at:
> http://bird.org/fuego/Unified_Results_Format_Project

Has anyone looked at the JSON schema we designed for kernelci.org[1]?  I
wasn't involved in the meetings or call, but from what I can see, one
important thing missing from your current proposal is how to group
related tests together.  In the schema we did for kCI, you can have test
cases grouped into test sets, which in turn can be grouped into a test
suite.  IMO, this is crucial since most tests come as part of a larger
suite.

The kernelci.org project has prototyped this for several testsuites
(kselftest, hackbench, lmbench, LTP, etc.) and were pushing JSON results
using this /test API to our backend for awhile.  But nobody got around
to writing the parsing, reporting stuff yet.

All of that to say the kCI JSON schema has been through through quite a
bit already, and acually used for several test suites, so I think it
would be a good idea to start with that and extend it, so combined
efforts on this JSON test schema could benefit Fuego as well as other
projects.

Kevin

[1] https://api.kernelci.org/schema-test.html
    https://api.kernelci.org/collection-test.html