[Fuego] showstopper for us

[Rafael Gago Castano]

Hello again.

Just now I showed my proof of concept with fuego to our lead dev. Unfortunately
he found (and I agree) one showstopper.

It's a bit related with how the fuego repositories are stored on the server.

On our devices locally we can't develop everything on our machines and then
push to the server, as the test rig has some device networks and external
hardware that we haven't on our machines.

This means that to develop a test we may need to do it directly on the server
filesystem, affecting other scheduled runs (by other users) if we happen to 
touch a common file (e.g. the board setup/teardown files). This should happen
rarely IMO.

This implies too that only one developer can develop tests on the test rig at
the same time (git commits), this is IMO the real showstopper.

How LAVA works is that it each test is a YAML file that contains information
that says where to git pull the test from (which I don't like as it has two 
levels of indirection).

Fullfilling this requirement would need severe refactoring in fuego.

Just to share my random ideas, If I were given an infinite time budget and the
ability to break as many compatibility as I wish I'd do so:

-Build nodes/board files: Similar to what they are now with all the parameters
 defined as jenkins variables available from the scripts. The build nodes would
 use jenkins labels as I wrote in my previous mail.

-Each node would have only two jenkins project named "Benchmark.run" and 
 "Functional.run" that would take two jenkins parameters. One that would serve
 as an URI:

 git=https://my_user_repo/;branch=master;sha=234df;path=tests/Functional.my_test"

 or for inbuilt fuego tests and modifications on the local filesystem (while
 developing):

 "path=engine/tests/Benchmark.java"

 And another one that would receive a KV list with all the parameters

 "BAUDRATES='9600 115200'; DEV="/dev/ttuSUB0"

Then the test would be fetched if they are non-local/git prefixed and the 
generation scripts run (may need some fuego pregeneration steps).

Then ftc would not be required to start jobs, just the jenkins CLI would do.

This is a very spontaneous idea that doesn't solve a lot of problems (reporting/
how to tell tests from each other, device setup/teardown etc).

So it seems like we abandon fuego for now, as we can't justify one-time 
refactors that big (suposing that everyone agreed to) to management when my
coleagues have already invested a lot of time getting the existing LAVA setup
working.

It's a shame, as I like the simplicity and the potential it has. Thank you all
for your help. I'm going to take a small vacation until the 15th, so I won't
be answering until then.

Thanks for all the help. Hopefully we will be able to revisit the project some
time. I hope that you find the feedback useful.