[Fuego] First steps with fuego

[Bird, Timothy]

> -----Original Message-----
> From: Rafael Gago Castano Friday, May 12, 2017 7:34 AM
> To: Bird, Timothy <Tim.Bird at sony.com>
> Cc: fuego at lists.linuxfoundation.org
> Subject: Re: First steps with fuego
> 
> As I have no infinite budget to investigate fuego, I decided to skip looking at
> that signal error on "ls" and to assume that we can do communication tests
> with fuego, it may need some refinements but there isn't any big barrier.

OK good.  Don't hesitate to ask questions.  We don't want people to burn
out trying to understand Fuego (which is admittedly a bit complicated),
and it's very helpful for us to see what issues come up and what
questions people have, so we can improve Fuego.

> As a matter of fact for our use case it's simpler than by using LAVA multinode
> tests.
That's good to hear.  Hopefully we can add some synchronization primitives
to match the functionality available with LAVA more easily.

> 
> Then I started looking at the device "power off + power off + flashing + wait
> for boot" procedure. I tried to do it as a test but it had a problem, when the
> device is rebooting it hasn't connectivity, so it does fail:
> 
> SDKTARGETSYSROOT=/opt/slp/dingo-next/sdk/sysroots/cortexa9hf-neon-
> oe-linux-gnueabi
> Target_PreCleanup=true
> _=/usr/bin/env
> ssh: connect to host 192.168.1.2 port 22: No route to host
> ssh: connect to host 192.168.1.2 port 22: No route to host
> 
> *** ABORTED ***
> 
> Fuego error reason: Cannot connect to 192.168.1.2 via ssh
> 
> in signal_handler
> ##### doing fuego phase: post_test ########
> 
> I guess that to do "power off + power off + flashing + wait for boot" would
> require a different approach than writing a test.
> 
> At first I thought about having two new types of tests: setup and teardown,
> but these aren't conceptually tests. IMO these belong to the board, so just
> adding an optional "setup" and "teardown" functions to the board file could
> be a good starting point.
> 
> Then the problem would be to decide when to run them. Running them as a
> test (of a new type) makes very explicit to see and control when they run.
> Having them on the board file would need rules or maybe flags for "ftc",
> because the setup/teardown functions should definitely run as the first and
> last actions of a testplan/batch, but they would be detrimental when
> developing tests.
It sounds like this is the best approach.  See below for some existing 
fuego functionality that might be useful for this.

> 
> Then the third approach would be to leave the board setup and teardown to
> the Jenkins side and let the user handle the integration. This makes the user
> to have more scattered configuration (configure boards in two places) but
> keeps Fuego simpler. I'm not very fluent  with Jenkins hacking but it seems
> doable too.
> 
> Which approach seems the best?

There are variables now, that you can define on a board, to handle
link setup and link teardown.  I think these could be also be used for
the purpose of system redeploy and board boot.

If you define TARGET_SETUP_LINK in your board file, as the name
of a function, then Fuego will call that function during the 'pre_test' phase.

This is not well-documented on the wiki (there's one obscure
reference to it on this page: http://bird.org/fuego/function_pre_test  )
The reason this is  not well-documented yet is that this functionality
is currently undergoing some change.  We are switching to using
explicit transport routines, which can be overridden in the board file.
That is, I've just added support for new routines ov_transport_connect
and ov_transport_disconnect that are intended to replace this
functionality.  However, we plan on leaving in support for this
functionality for legacy reasons, and you should be able to use the
current technique to do anything you want to get the board
operational and ready for network communication (including build,
system deploy, and boot), in your own custom routine.

Here's an example (I haven't tested):
You put this in your <board>.board file for the DUT.

function my_custom_board_bringup {
  echo "do a bunch of interesting stuff here"
}
TARGET_SETUP_LINK="my_custom_board_bringup"

By default, things defined in the board file will be executed for
every test. However, you might make the setup conditional
by testing some condition inside your custom setup function, to decide
when the operation is needed.  For example, maybe you could set
a variable in a spec file that is examined in the custom routine to decide
whether to skip it.

Note that similar functionality for link teardown (or board shutdown)
is available using TARGET_TEARDOWN_LINK, which is called in the
post_test phase by Fuego.

I hope this addresses what you need.
 -- Tim