[lsb-discuss] LSB 3.2 additions to "desktop" test suite

Alexey Khoroshilov khoroshilov at ispras.ru
Fri Apr 13 10:02:22 PDT 2007


>> -----Original Message-----
>> From: lsb-discuss-bounces at lists.freestandards.org 
>> [mailto:lsb-discuss-bounces at lists.freestandards.org] On Behalf 
>> Of Wichmann, Mats D
>> Sent: Friday, April 13, 2007 6:45 AM
>> To: Alexey Khoroshilov
>> Cc: lsb-discuss
>> Subject: Re: [lsb-discuss] LSB 3.2 additions to "desktop" test suite
>>
>>
>>>> In general, there seem two approaches we could take:
>>>> (1) redesign the test in some way so that it can be
>>>> operated non-interactively (could include using some
>>>> sort of driver which handles the interactive parts)
>>>> (2) split this test out into the small list (all the
>>>> others are currently appbat applications) which can
>>>> not be driven from the test driver.
>>> At the first glance all interactions of the freedesktop.org tests are
>>> important to make decision on correctness of the system under
>>> test. So I believe the second approach is more attractive.
>>>
>>> But it would be good to split out really interactive tests 
>> only and to
>>> keep non-interactive part of the test suite in the lsb-test-desktop.
>> The
>>> reason is as follows. The LSB Distribution Test Kit supports 
>> two basic
>>> use cases: certification test run and "nightly" regression test run.
>> The
>>> first one includes all interactive tests, while the second one
>>> requires fully automated execution.
>>>
>>> And we would like to keep all non-interactive tests in the "nightly"
>>> test execution and to have additional interactive tests for bonus and
>>> certification. 
>>>
>>> This can be achieved if the lsb-test-desktop takes a parameter, which
>>> controls the set of tests to be executed:
>>> - non-interactive only;
>>> - interactive only;
>>> - both.
>> This wouldn't be too hard to add.
>>
>>> The parameter will help to the lsb-dtk-manager to manage test
>>> execution. For "nightly" test runs the lsb-dtk-manager will execute
>>> all non-interactive tests. For certification test runs the
>>> lsb-dtk-manager will execute sequentially: at first all
>>> non-interactive tests, and then all interactive tests.
>> The placement of the fdo test at the beginning of desktop was
>> to implement a similar concept, but inverted - I believe the
>> thinking was you'd kick off the test suite, handle the interactive
>> bits right away, and then go off and let the rest run by itself.
>> (This was contributed so I don't know the exact rationale)
> 
> Yes, that was the rationale.
> 

Our rationale is as follows. We suppose that non-interactive tests can 
detect some problems, which requires changes in the system under test. 
If we perform manual tests early and some changes in distro are needed, 
one will be forced to repeat the manual work again.

Otherwise the manual work will be performed only if one is confident in 
successful finish of automated part.

Of course lsb-dtk-manager can support both orders.




More information about the lsb-discuss mailing list