[Fuego] showstopper for us
daniel.sangorrin at toshiba.co.jp
Fri Jun 2 23:28:30 UTC 2017
> -----Original Message-----
> From: fuego-bounces at lists.linuxfoundation.org [mailto:fuego-bounces at lists.linuxfoundation.org] On Behalf Of Rafael Gago Castano
> Sent: Friday, June 02, 2017 11:26 PM
> To: fuego at lists.linuxfoundation.org
> Subject: [Fuego] showstopper for us
> Hello again.
> Just now I showed my proof of concept with fuego to our lead dev. Unfortunately
> he found (and I agree) one showstopper.
> It's a bit related with how the fuego repositories are stored on the server.
> On our devices locally we can't develop everything on our machines and then
> push to the server, as the test rig has some device networks and external
> hardware that we haven't on our machines.
> This means that to develop a test we may need to do it directly on the server
> filesystem, affecting other scheduled runs (by other users) if we happen to
> touch a common file (e.g. the board setup/teardown files). This should happen
> rarely IMO.
> This implies too that only one developer can develop tests on the test rig at
> the same time (git commits), this is IMO the real showstopper.
Mmm sorry I think I didn't get the whole picture here. Is it something like this?
[User1's Desktop Fuego]---+
|----[Server with LAVA]---[DUT(device under test)/test rig?]
[User2's Desktop Fuego]---+
> How LAVA works is that it each test is a YAML file that contains information
> that says where to git pull the test from (which I don't like as it has two
> levels of indirection).
> Fullfilling this requirement would need severe refactoring in fuego.
At the moment test's source code can be pulled from a git repository just before building.
We are also planning to add functionality for installing a test (parser+fuego_test.sh+spec) from a git repository:
ftc install-test https://github.com/xx/mytest.git
> Just to share my random ideas, If I were given an infinite time budget and the
> ability to break as many compatibility as I wish I'd do so:
> -Build nodes/board files: Similar to what they are now with all the parameters
> defined as jenkins variables available from the scripts. The build nodes would
> use jenkins labels as I wrote in my previous mail.
> -Each node would have only two jenkins project named "Benchmark.run" and
> "Functional.run" that would take two jenkins parameters. One that would serve
> as an URI:
> or for inbuilt fuego tests and modifications on the local filesystem (while
> And another one that would receive a KV list with all the parameters
> "BAUDRATES='9600 115200'; DEV="/dev/ttuSUB0"
> Then the test would be fetched if they are non-local/git prefixed and the
> generation scripts run (may need some fuego pregeneration steps).
> Then ftc would not be required to start jobs, just the jenkins CLI would do.
> This is a very spontaneous idea that doesn't solve a lot of problems (reporting/
> how to tell tests from each other, device setup/teardown etc).
Yeah, I think this is a completely different approach. We want to focus on
an easy-to-use tool that anyone can use for running tests. Having only two
jobs would complicate the visualization graphics, download links, comparisons
between test results, etc..
> So it seems like we abandon fuego for now, as we can't justify one-time
> refactors that big (suposing that everyone agreed to) to management when my
> coleagues have already invested a lot of time getting the existing LAVA setup
> It's a shame, as I like the simplicity and the potential it has. Thank you all
> for your help. I'm going to take a small vacation until the 15th, so I won't
> be answering until then.
> Thanks for all the help. Hopefully we will be able to revisit the project some
> time. I hope that you find the feedback useful.
I'm sorry to hear that Fuego didn't fit your use case. I really appreciate all of your
comments and feedback. I hope you have a good vacation :)
More information about the Fuego