[Fuego] Status of LTP parser - populating run.json
Tim.Bird at sony.com
Thu Aug 10 01:10:59 UTC 2017
> -----Original Message-----
> From: Daniel Sangorrin on Wednesday, August 09, 2017 5:49 PM
> > -----Original Message-----
> > From: Bird, Timothy [mailto:Tim.Bird at sony.com]
> > I've been doing some testing with LTP and the new parser. I'm only seeing
> one result show up
> > in the run.json file. This is due to a bug in the process_data routine. When
> I made it backwards
> > compatible with the other functional tests, I broke it for LTP.
> Don't worry, it was broken already by my changes and waiting for an
> > I think it would be better to move to the LTP parser.py program calling
> plib.process(), rather than
> > process_data. Currently, only old-style functional tests call that routine,
> with a single value
> > provided by generic-parser.py, and old-style benchmark tests, with values
> that can be made
> > into measurements.
> Yes, definitely.
It's fixed in my 'next' branch - at least LTP and other tests work for me
(populating run.json with testcase status).
> Thanks for the fixes.
> Now that the parser seems to work for many tests, it would be good to
> upgrade the LTP to support
> the new parser natively. The script already uses "test_category" which will
> become "test_set" and
> "test_case" which has a direct equivalent. I hope it will not be too much
I didn't change the variable names in LTP's parser.py, but I think it
does what we want now. Actually, looking at it just now, I'm fine leaving
it as "test_category". I'm not sure exactly what the LTP nomenclature
for the thing is - I've seen the wording "test scenario" used.
LTP is a bit weird, they have multiple test defined in runtests which can call
lots of sub-tests. I found many different test scenarios (if that's the right phrase)
that call syscall-related test programs. So these things can be mixed and matched.
I thought about adding the sub-test results as "measures" under the test cases,
but I'm holding off until I make more progress on the report generation.
As I'm working through things, there are a few items I might like to propose
for the schema, but overall things seem to be working well.
More information about the Fuego