[Fuego] Unification of functional and benchmark tests

Bird, Timothy Tim.Bird at sony.com
Thu Apr 27 00:14:53 UTC 2017



> -----Original Message-----
> From: Daniel Sangorrin on Monday, April 24, 2017 1:38 AM
> These patches contain a big change that unifies functional and
> benchmark tests. These are the main points but for more detailed
> fixes please check the source code.
> 
> - functional.sh and benchmark.sh have been merged into main.sh which
> is called directly from Jenkins (ftc needs update).
I've made a todo for this.

>   + TODO: rename bc.sh to fuego_test.sh (for all tests)
Also still on my todo list.  If we're serious about this, it should happen
this release (1.2).  I don't want to go through another major
refactoring after the 1.2 release, as I'm hoping that after 1.2 I
can start promoting a usage model where tests live outside the
fuego-core repository.  Once that happens, we'll need to have backwards
compatibility, so major refactoring will be more difficult.

> - Both type of tests (Functional and Benchmark) now output a
> "results.json" file in the same format. When I implement "ftc report",
> all those results.json files will be combined to create a pdf/html/excel report.
Sounds good!

>   + TODO: runs.json information needs to be merged into results.json
>     and we should make sure that we can generate KernelCI requests as well.
Agreed.
  -- Tim

> 
> fuego patches:
> [PATCH] flot: unify functional and benchmark
> 
> fuego-core patches:
> [PATCH 1/3] core: the great unification of functional and benchmark
> [PATCH 2/3] fix expat test and add it to testplan docker
> [PATCH 3/3] add Functional.jpeg test to testplan docker since it
> 
> Next steps
>    - Parse LTP to produce json instead of an excel sheet
>    - Add functionality for blacklisting test cases
Yes!  This is highly needed.

>    - merge runs.json information into results.json
>    - implement ftc report

Sounds like a good list.  Let me know what I can help with.
 -- Tim


More information about the Fuego mailing list