[Fuego] Fuego criteria file documentation
Bird, Timothy
Tim.Bird at sony.com
Tue Nov 14 01:53:01 UTC 2017
> -----Original Message-----
> From: fuego-bounces at lists.linuxfoundation.org [mailto:fuego-
> bounces at lists.linuxfoundation.org] On Behalf Of Liu, Wenlong
> Sent: Monday, November 13, 2017 2:38 AM
> To: fuego at lists.linuxfoundation.org
> Subject: Re: [Fuego] Fuego criteria file documentation
>
>
> > -----Original Message-----
> > From: fuego-bounces at lists.linuxfoundation.org
> > [mailto:fuego-bounces at lists.linuxfoundation.org] On Behalf Of Bird,
> > Timothy
> > Sent: Thursday, November 02, 2017 8:53 AM
> > To: fuego at lists.linuxfoundation.org
> > Subject: [Fuego] Fuego criteria file documentation
> >
> > Hello all (but especially Daniel),
> >
> > I've been writing the documentation for new features in the Fuego 1.2
> release,
> > and I've put together a page about the new criteria file.
> >
> > The page is here:
> > http://fuegotest.org/wiki/criteria.json
> >
> > Can you please take a look and see the information there is correct?
> I have seen the current Fuego criteria documentation.
> One question as below.
>
> As shown in the example of this documentation,
> {
> "schema_version":"1.0",
> "criteria": [
> {
> "tguid": "syscall",
> "min_pass": 1000,
> "max_fail": 5
> },
> {
> "tguid": "timers",
> "fail_ok_list": ["leapsec_timer"]
> },
> {
> "tguid": "pty",
> "must_pass_list": ["hangup01"]
> }
> ]
> }
> Yes, the example can works well.
> However, you know, for LTP test, there are more than 70 sub-testsuites.
>
> For sub-testsuite named "net.nfs", the criteria json file cannot works as I
> expected even though I added "net.nfs" as above.
> When function "split_test_id(test_case_id)" was executed, the testset name
> was changed to "default" and we cannot find the correct criteria json file.
> The reason might be that "net.nfs" has a character ".".
That was indeed the cause. The full test_case_id included multiple
periods, which split_test_id did not handle correctly.
>
> So, any advice for this situation or it should be fixed?
I corrected this routine, with commit: f50cff2 in the master branch.
However, I didn't try to use a criteria file with with net.nfs. Can you
check out the latest code and try again?
>
> PS:
> * Attached is my criteria json file for AGL, although it cannot works as I
> expected(reference only).
> (LTP Version:20170516; Target: R-car M3)
> * I didn't list all network related tests in this criteria json file because of the
> issue above.
Thanks for the bug report.
-- Tim
More information about the Fuego
mailing list