[Fuego] Unification of functional and benchmark tests
Bird, Timothy
Tim.Bird at sony.com
Thu Apr 27 00:14:53 UTC 2017
> -----Original Message-----
> From: Daniel Sangorrin on Monday, April 24, 2017 1:38 AM
> These patches contain a big change that unifies functional and
> benchmark tests. These are the main points but for more detailed
> fixes please check the source code.
>
> - functional.sh and benchmark.sh have been merged into main.sh which
> is called directly from Jenkins (ftc needs update).
I've made a todo for this.
> + TODO: rename bc.sh to fuego_test.sh (for all tests)
Also still on my todo list. If we're serious about this, it should happen
this release (1.2). I don't want to go through another major
refactoring after the 1.2 release, as I'm hoping that after 1.2 I
can start promoting a usage model where tests live outside the
fuego-core repository. Once that happens, we'll need to have backwards
compatibility, so major refactoring will be more difficult.
> - Both type of tests (Functional and Benchmark) now output a
> "results.json" file in the same format. When I implement "ftc report",
> all those results.json files will be combined to create a pdf/html/excel report.
Sounds good!
> + TODO: runs.json information needs to be merged into results.json
> and we should make sure that we can generate KernelCI requests as well.
Agreed.
-- Tim
>
> fuego patches:
> [PATCH] flot: unify functional and benchmark
>
> fuego-core patches:
> [PATCH 1/3] core: the great unification of functional and benchmark
> [PATCH 2/3] fix expat test and add it to testplan docker
> [PATCH 3/3] add Functional.jpeg test to testplan docker since it
>
> Next steps
> - Parse LTP to produce json instead of an excel sheet
> - Add functionality for blacklisting test cases
Yes! This is highly needed.
> - merge runs.json information into results.json
> - implement ftc report
Sounds like a good list. Let me know what I can help with.
-- Tim
More information about the Fuego
mailing list