[Fuego] [PATCH] Fuego Release Test Repository

Gustavo Sverzut Barbieri barbieri at profusion.mobi
Tue Feb 20 14:07:04 UTC 2018


On Mon, Feb 19, 2018 at 10:25 PM,  <Tim.Bird at sony.com> wrote:
>
>
>> -----Original Message-----
>> From: Gustavo Sverzut Barbieri
>> On Fri, Feb 16, 2018 at 7:42 PM, Guilherme Campos Camargo
>> <guicc at profusion.mobi> wrote:
>> ...
>> > Also, it's important to note that this patch represents a sort of
>> > paradigm shift in the way that tests are added to Fuego, given that:
>> > - The software requirements are being installed through a Dockerfile
>> >   that extends fuego-base.
>> > - The test routines/plans and the new boards are moved to Fuego during
>> >   build time, acting almost as an "overlay".
>> > For that reason, it's imperative that current users/contributors provide
>> > their feedback regarding this new approach before we move on. Also,
>> > since I just started contributing with and using fuego, it's likely that
>> > I missed something, specially related to Fuego usability. So please,
>> > kindly add your comments and correct me if so.
>>
>> Tim, I discussed this with Guilherme and we'll formalize a proposal
>> around the spec.json file, something along the lines:
>>
>> 1) test specify the debian repositories, packages and even gpg/keys
>> (for repos). Instead of always adding the packages to the container,
>> either by modifying the official docker image (as done before) or
>> creating an overlay (as done by Guilherme), Fuego would make sure the
>> repositories are added and packages are added prior to executing the
>> tests.
>>
>
> Since I haven't been able to run the release-test yet, this is also going
> over my head.
>
> What problem is this solving?
>
> Is this for a test to specify additional packages for the base distribution
> in the docker container, that the test needs on the host?
>
> Or is it referring to a new -dev package used for building the software
> for the target (but again, something that needs to be in the
> docker container, but is part of the toolchain)?
>
> Would this be used for something like:
>  lib-cairo-dev, or netperf-server?

a bit of all, these are dependencies *inside the fuego container* to
build and run the tests. (nothing is installed in the user host
machine).

so for tests that needs libcairo-dev to build/run, that would be the
dependencies.

for tests that need the netperf-server, that would be the dependencies.

for fuego-rt, we need docker (client inside the container to contact
the host to build images), also need python's pexpect, selenium and so
on.

for cross-compiled tests, it would be the cross compiler toolchain.
However, for toolchains/cross-compile, things may be better defined
per target board/architecture, not per test...




>> JSON object would look like:
>>
>>    "packages-dependencies": {
>>        "repositories": ["deb https://download.docker.com/linux/debian
>> jessie"],
>>        "keys": ["https://download.docker.com/linux/debian/gpg"],
>>        "packages": ["docker-ce", "libxpto-dev"],
>>    }
>>
>> Which would manage /etc/apt/sources.d/fuego.conf + apt-get install
>>
>>
>> 2) in addition to the test scenarios variables that are pre-defined,
>> allow user to redefined/override those when executing the test using
>> the command line. For example fuego-release-test (fuego-rt) would
>> default to test official repository with the branch "next", but one
>> could override to test a private repo with a custom branch.
>>
>> This would solve the need to "patch" the correct value at entrypoint.sh:
>>
>>    sed -i "s/FUEGOTEST_REPO/${test_repo_escaped}/"
>> /fuego-core/engine/tests/Functional.fuegotest/spec.json
>>    ...
>>    ftc add-jobs -b fuego-test -t Functional.fuegotest
>>
>> With something like:
>>
>>    ftc add-jobs -b fuego-test -t Functional.fuegotest -v
>> FUEGOTEST_REPO=${test_repo_escaped}
> We've talked about adding the ability to define a test
> variable on the command line, when adding jobs
> or running tests, that would override the value contained
> in a spec file.
>
> This is highly desirable.
>
> I'm still, however, wrapping my head around the fuego release
> self-test case, which I don't understand yet.  Talking
> about this feature in that context may be too much for my
> little brain to handle, at the moment.

the big picture is:

your-linux-pc (aka: desktop)
  -> runs fuego-stable (aka: stable-container)
     -> executes functional test fuego-release-test (ftc add-jobs -b
fuego-test -t Functional.fuegotest)
         - download fuego from $FUEGOTEST_REPO + $FUEGOTEST_BRANCH
         - generate a new docker image for that
         - runs a container for that image
         - executes release tests for that container (ie: calls "ftc"
for that container)
         - remove that container and image
         - reports back test results
     <- report if fuego-release-test passed/failed

then, the parameters here (git repo and branch) must be passed from
your desktop to fuego-stable so it knows what is to be tested. We can
provide few fixed test scenarios, such as official git repo and
branches: master + next.

but say I have my own fork and I want to test that repo+branch, I need
a way to specify that.

What Guilherme did to address that is to dyamically patch the
spec.json, which is bad (just stop-gap so he can test with his own
repo).


-- 
Gustavo Sverzut Barbieri
http://profusion.mobi embedded systems
--------------------------------------
MSN, GTalk, FaceTime: barbieri at gmail.com
Skype: gsbarbieri
Mobile: +55 (16) 99354-9890


More information about the Fuego mailing list