[Fuego] UART/USB Connectivity, History Database, Test result format

[Bird, Timothy]

> -----Original Message-----
> From: dhinakar k on Tuesday, March 28, 2017 7:34 PM
> 
> Hi Tim,
> 
> That was a very detailed reply.
> Thank you very much.
> 
> Just one follow up question.
> How does Fuego compare with Avocado?
> If you had checked it out already please let me know.

I did take a quick look at Avacado a while back, and put
some notes at:
http://bird.org/fuego/Other_test_systems#Avacado

Avacado has some very nice features.
It seems like it excels at handling large test matrices.  It has yaml
files to define the different test variations.

They also interface well with Jenkins - a single command-line from a Jenkins
job can start an avocado test, and they product output that is easily parsed
by existing Jenkins post-processors.
e.g. 'avacado run /bin/true sleeptest passtest --xunit "$WORKSPACE/results.xml"...'

They have an 'avacado server' concept, where each host can execute jobs from
another machine (I believe - I haven't tried it out).

So far, it looks like Avacado is used for virtualization testing.  I don't see a 
lot of different tests that run on this framework.

Here are some of the major difference, that I can see (this is with limited
study of Avacado, so I apologize for any omissions of features on their part):
1) Fuego is both a test framework, and a distribution of tests themselves.
  * We have about 60 tests now, with about 90 in our 'next' branch
     * the goal is to have hundreds of tests available, for a wide variety of system features and issues
2) Fuego provides a consistent, containerized back end (docker) so that the
 builds of the test software can be uniform between different sites.
   * this will help when comparing results from different sites
       * multi-site results comparisons are not done yet, but on our roadmap
3) Fuego has a multi-node test API (host/target), that is specially geared for
embedded linux testing
  * the footprint on the target is intentionally very small, using existing POSIX features
     * IOW - there is no specialized target agent on the target board (except for a transport agent, like sshd or adb)
  * Fuego is geared towards testing of final production images of the software stacks for products
4) Fuego is behind Avacado in terms of:
   a) unified test ouput (we're working on that now)
   b) test results sharing
       * (also in-progress - 'runs' can be packaged and shared now to a central server, in our 'next' branch)
       * however, there's only rudimentary test display and analysis on the Fuego server at the moment
           * Avacado seems to be using Jenkins for this - and we should look at them more closely to see what they're doing

There are more, but hopefully this is helpful to see some of the differences.
 -- Tim