[Fuego] Fuego criteria file documentation

Bird, Timothy Tim.Bird at sony.com
Tue Nov 14 01:53:01 UTC 2017



> -----Original Message-----
> From: fuego-bounces at lists.linuxfoundation.org [mailto:fuego-
> bounces at lists.linuxfoundation.org] On Behalf Of Liu, Wenlong
> Sent: Monday, November 13, 2017 2:38 AM
> To: fuego at lists.linuxfoundation.org
> Subject: Re: [Fuego] Fuego criteria file documentation
> 
> 
> > -----Original Message-----
> > From: fuego-bounces at lists.linuxfoundation.org
> > [mailto:fuego-bounces at lists.linuxfoundation.org] On Behalf Of Bird,
> > Timothy
> > Sent: Thursday, November 02, 2017 8:53 AM
> > To: fuego at lists.linuxfoundation.org
> > Subject: [Fuego] Fuego criteria file documentation
> >
> > Hello all (but especially Daniel),
> >
> > I've been writing the documentation for new features in the Fuego 1.2
> release,
> > and I've put together a page about the new criteria file.
> >
> > The page is here:
> > http://fuegotest.org/wiki/criteria.json
> >
> > Can you please take a look and see the information there is correct?
> I have seen the current Fuego criteria documentation.
> One question as below.
> 
> As shown in the example of this documentation,
> {
>     "schema_version":"1.0",
>     "criteria": [
>     {
>         "tguid": "syscall",
>         "min_pass": 1000,
>         "max_fail": 5
>     },
>     {
>         "tguid": "timers",
>         "fail_ok_list": ["leapsec_timer"]
>     },
>     {
>         "tguid": "pty",
>         "must_pass_list": ["hangup01"]
>     }
>     ]
> }
> Yes, the example can works well.
> However, you know, for LTP test, there are more than 70 sub-testsuites.
> 
> For sub-testsuite named "net.nfs", the criteria json file cannot works as I
> expected even though I added "net.nfs" as above.
> When function "split_test_id(test_case_id)" was executed, the testset name
> was changed to "default" and we cannot find the correct criteria json file.
> The reason might be that "net.nfs" has a character ".".

That was indeed the cause.  The full test_case_id included multiple
periods, which split_test_id did not handle correctly.

> 
> So, any advice for this situation or it should be fixed?
I corrected this routine, with commit: f50cff2 in the master branch.

However, I didn't try to use a criteria file with with net.nfs.  Can you
check out the latest code and try again?

> 
> PS:
> * Attached is my criteria json file for AGL, although it cannot works as I
> expected(reference only).
>   (LTP Version:20170516; Target: R-car M3)
> * I didn't list all network related tests in this criteria json file because of the
> issue above.

Thanks for the bug report.
 -- Tim



More information about the Fuego mailing list