[Fuego] Some problems at agl testplan

[Bird, Timothy]

> -----Original Message-----
> From: Cai, Song on Friday, September 15, 2017 1:45 AM
> Hi all
> 
> I'm running test case of Fuego's testplan agl at branch next on db410c.
> 
> And I meet some problems and try to solve them. But the following three
> ones puzzle me. Hope for some help.
> 
> 1.     Functional.libspeex   Functional.libogg
> 
> both script fuego_test.sh and the content of spec tarball of them are same.
> Why?
I don't know.  To be honest some of these tests I have never run, because my
boards don't have the associated software.

It looks to me like this was a copy-and-paste that didn't get finished.  In
Functional.libogg/fuego_test.sh there is no testing of libogg that I can
see.  Since it doesn't add anything, I would say we should remove
Functional.libogg.

However, I'm copying Huang Feng in case he has some comment.

> 
> 
> 
> 2.     Functional.libtar
> 
> The line 12 at fuego_test.sh as following:
> 
> libtar -t testfile.tar | grep testfile
> 
> 
> 
> libtar is a library, but it's used as an command here. Is it just a bug, or libtar
> can be compiled from some package?

I think it's a bug.  I am not familiar with any use of libtar as a command.
Looking at it, it's a pretty simple test, but I guess it verifies if 'tar' is present
and can be used to create a tar, list tar contents and extract the content.
The test could be improved by checking that the extracted contents matched
the original contents put into the tar, and testing different compression options.

> 
> 
> 
> 3.     The case about commonAPI :
> 
> Functional.commonAPI_C++
> 
> Functional.commonAPI_Dbus
> 
> Functional.commonAPI_SomeIp
> 
> 
> 
> There are only test codes at those three case, without any library.
> 
> The code is generated by commonapi generator, which depends on the
> specified version.
> 
> But no info about the commonapi version is avaliable.
> 
> So, can we provide fidl and fdepl file directly?
Are there required for the build or on the target (or both)?
I don't know much about the commonAPI stuff, but it looks
like it's a Genivi project.

There are a few routes we could  take here:
1) add a dependency on fidl and fdepl to the package - check for them in 
test_pre_check and if they are not present, abort the test.
This could be done using is_on_target or is_on_sdk and assert_define.

2) add fidl and fdepl to the package tar and use them during the build

3) add fidl and fdepl to the docker build, and use them during the build

In options 2 and 3, you still have to worry about whether the libraries are
there on the target.

My preference is option 1).  If we're not testing with an SDK with the required
build libraries or a target board with the required runtime libraries, it seems like
the software (and test) is actually out-of-scope for the environment we're testing.

This means that you would only run this test on a board, and with an SDK that
had those libraries.  The failure should be noted in case people run the testplan_agl
with other board expecting everything to succeed.

Thanks for looking into this and reporting these problems.
 -- Tim