[agl-discussions] proposal for using LTSI test tool(based on Jenkins)

[qiaonuohan]

Hi Jeremiah,

Thanks for your reply.

On 09/22/2015 04:13 PM, Jeremiah Foster wrote:
> On Tue, Sep 22, 2015 at 9:37 AM, qiaonuohan <qiaonuohan at cn.fujitsu.com <mailto:qiaonuohan at cn.fujitsu.com>> wrote:
>
>     Hello AGL people,
>
>
> Thank you for joining the discussion qiaonuohan.
>
>     After checking some mails[1] in mail list, I find it is time to build test
>     framework. We fujitsu ten suggests using LTSI test tool(also known as JTA).
>
>     JTA is wildly used with many cases focusing on LTSI. Since AGL uses LTSI as
>     its kernel, why not choose JTA when we want to get kernel tested, especially
>     some cases related to LTP.
>
>
> The LTSI test tool looks like a good approach. There are some drawbacks, like having to download the entire tarball and configure before you can integrate a test. If we were to use this it would likely have to be hosted somewhere. Other issues remain, like integration into LAVA which also has a large set of tests.

If I failed to understand this, please forgive me. You mean it's better to move the test
cases into the AGL distro, then bitbake-ize them?

JTA doesn't need a lot of effort to configure. Actually, in our daily work, we feel it
saves a lot of effort on configuration with many boards need to be tested.


Sorry about LAVA, could you give some hint about this?

>
>     And fujitsu ten has also developed many in-house testcase based on JTA, which
>     contains tests on kernel, driver, middle ware, API and etc. It will be easier
>     for us to push these cases to AGL, if JTA is chosen, and help to save AGL
>     people a lot of effort.
>
>
> In my experience having running code tends to bring contributors more effectively than just having a test tool, the in house test cases really need to be released as soon as possible to persuade others to contribute.
>
>     Recently, we have started trying to offer JTA available to work on AGL distro +
>     porter, and get some cases, of course based on JTA，worked. If any people are
>     interested in it, we can work together to get AGL test framework ready.
>
>
> What do you propose to be the next steps for collaboration? Are there specific areas you need to test? Is there a list of tests you have in mind or that AGL should compile?

The first thing we want to do is to move LTP related cases in JTA into AGL. Then we
would like to push our in-house cases in AGL, and its amount is about 800 and most related
to middleware/pkg, like busybox, gstreamer and etc. Also I think other members do have
their own test case. It will be much better to choose this wildly used tool to get all
of these cases merged.

OTOH, when we forwarding AGL, there must be some problem occurred in LTSI, we need fix
and test it in AGL first then feedback to LTSI. If AGL and LTSI using the same test tool,
things will be much easier.

>
> Regards,
>
> Jeremiah
>
>
>     [1]
>     Currently, two threads talked about test:
>     1. http://lists.linuxfoundation.org/pipermail/automotive-discussions/2015-September/000819.html
>     2. http://lists.linuxfoundation.org/pipermail/automotive-discussions/2015-September/000822.html
>
>
>
>     --
>     Regards
>     Qiao Nuohan
>
>
>
>
> --
> Jeremiah C. Foster
> GENIVI COMMUNITY MANAGER
>
> Pelagicore AB
> Ekelundsgatan 4, 6tr, SE-411 18
> Gothenburg, Sweden
> M: +46 (0)73 093 0506
> jeremiah.foster at pelagicore.com <mailto:jeremiah.foster at pelagicore.com>
>
>


-- 
Regards
Qiao Nuohan