[Fuego] Status of LTP parser - populating run.json

Bird, Timothy Tim.Bird at sony.com
Thu Aug 10 01:10:59 UTC 2017

> -----Original Message-----
> From: Daniel Sangorrin on Wednesday, August 09, 2017 5:49 PM
> > -----Original Message-----
> > From: Bird, Timothy [mailto:Tim.Bird at sony.com]
> > I've been doing some testing with LTP and the new parser. I'm only seeing
> one result show up
> > in the run.json file.  This is due to a bug in the process_data routine.  When
> I made it backwards
> > compatible with the other functional tests, I broke it for LTP.
> >
> Don't worry, it was broken already by my changes and waiting for an
> upgrade.
> > I think it would be better to move to the LTP parser.py program calling
> plib.process(), rather than
> > process_data.  Currently, only old-style functional tests call that routine,
> with a  single value
> > provided by generic-parser.py, and old-style benchmark tests, with values
> that can be made
> > into measurements.
> Yes, definitely.

It's fixed in my 'next' branch - at least LTP and other tests work for me
(populating run.json with testcase status).


> Thanks for the fixes.
> Now that the parser seems to work for many tests, it would be good to
> upgrade the LTP to support
> the new parser natively. The script already uses "test_category" which will
> become "test_set" and
> "test_case" which has a direct equivalent. I hope it will not be too much
> work.

I didn't change the variable names in LTP's parser.py, but I think it
does what we want now.  Actually, looking at it just now, I'm fine leaving
it as "test_category".  I'm not sure exactly what the LTP nomenclature
for the thing is  - I've seen the wording "test scenario" used.

LTP is a bit weird, they have multiple test defined in runtests which can call
lots of sub-tests.  I found many different test scenarios (if that's the right phrase)
that call syscall-related test programs.  So these things can be mixed and matched.

I thought about adding the sub-test results as "measures" under the test cases,
but I'm holding off until I make more progress on the report generation.

As I'm working through things, there are a few items I might like to propose 
for the schema, but overall things seem to be working well.
 -- Tim

More information about the Fuego mailing list