[Fuego] [PATCH 18/29] dbench: adapt to new parser

[Daniel Sangorrin]

> -----Original Message-----
> From: Bird, Timothy [mailto:Tim.Bird at sony.com]
> Sent: Tuesday, July 25, 2017 9:36 AM
> To: Daniel Sangorrin; fuego at lists.linuxfoundation.org
> Subject: RE: [Fuego] [PATCH 18/29] dbench: adapt to new parser
> 
> 
> 
> > -----Original Message-----
> > From: Daniel Sangorrin on Friday, July 21, 2017 1:02 AM
> >
> > Signed-off-by: Daniel Sangorrin <daniel.sangorrin at toshiba.co.jp>
> > ---
> >  engine/tests/Benchmark.dbench/criteria.json  | 16 ++++++++++++++++
> >  engine/tests/Benchmark.dbench/parser.py      | 19 +++++++++----------
> >  engine/tests/Benchmark.dbench/reference.json | 18
> > ++++++++++++++++++
> >  engine/tests/Benchmark.dbench/reference.log  |  3 ---
> >  4 files changed, 43 insertions(+), 13 deletions(-)
> >  create mode 100644 engine/tests/Benchmark.dbench/criteria.json
> >  create mode 100644 engine/tests/Benchmark.dbench/reference.json
> >  delete mode 100644 engine/tests/Benchmark.dbench/reference.log
> >
> > diff --git a/engine/tests/Benchmark.dbench/criteria.json
> > b/engine/tests/Benchmark.dbench/criteria.json
> > new file mode 100644
> > index 0000000..0c75391
> > --- /dev/null
> > +++ b/engine/tests/Benchmark.dbench/criteria.json
> > @@ -0,0 +1,16 @@
> > +{
> > +    "schema_version":"1.0",
> > +    "criteria":[
> > +        {
> > +            "tguid":"default.dbench.Throughput",
> > +            "reference":{
> > +                "threshold":0,
> > +                "comparison":"gt"
> > +            }
> > +        },
> > +        {
> > +            "tguid":"default.dbench",
> > +            "min_pass":1
> > +        }
> > +    ]
> > +}
> > diff --git a/engine/tests/Benchmark.dbench/parser.py
> > b/engine/tests/Benchmark.dbench/parser.py
> > index 9201075..67cf56c 100755
> > --- a/engine/tests/Benchmark.dbench/parser.py
> > +++ b/engine/tests/Benchmark.dbench/parser.py
> > @@ -1,18 +1,17 @@
> > -#!/bin/python
> > +#!/usr/bin/python
> >  # See common.py for description of command-line arguments
> >
> > -import os, re, sys
> > +import os, sys
> >
> >  sys.path.insert(0, os.environ['FUEGO_CORE'] + '/engine/scripts/parser')
> >  import common as plib
> >
> > -ref_section_pat = "\[[\w]+.[gle]{2}\]"
> > -cur_search_pat =
> > re.compile("^(Throughput)(.*)(MB/sec)(.*)(procs)$",re.MULTILINE)
> > +measurements = {}
> >
> > -cur_dict = {}
> > -pat_result = plib.parse(cur_search_pat)
> > -if pat_result:
> > -	print pat_result
> > -	cur_dict["Throughput"] = pat_result[0][1]
> > +regex_string = '^(Throughput)(.*)(MB/sec)(.*)(procs)$'
> > +matches = plib.parse(regex_string)
> >
> > -sys.exit(plib.process_data(ref_section_pat, cur_dict, 's', 'MBS'))
> > +if matches:
> > +    measurements['default.dbench'] = [{"name": "Throughput", "measure" :
> > float(matches[0][1])}]
> 
> I think these can be simplified.  I'm going to accept these new parsers as is,
> but think about how to simplify them.  Since the units are stored
> in the reference.json file, we may only need to pass a simple dictionary
> called 'measurements' with tguid as key and measurement value as the value
> back to process_data.
> 
> I want the parser.py module to be as simply as possible, because testers will
> be writing those.

About the units, that is actually what this patch does. Before you had to pass
the units (MBS) to process_data. Now, the parser grabs the units directly from
reference.json. Maybe you were confused with the regular expression?

Measurements is a dictionary where the key represents the tguid
of a test_case. In case of a Functional test, the value is the test_case's status string (PASS, FAIL..).
In case of a Benchmark test, the value is a list of measurements (or a combination).
I can move the measurement "name" to the tguid if that's what you mean.

Thanks,
Daniel

> 
> > +
> > +sys.exit(plib.process_data(measurements))
> > diff --git a/engine/tests/Benchmark.dbench/reference.json
> > b/engine/tests/Benchmark.dbench/reference.json
> > new file mode 100644
> > index 0000000..f08c750
> > --- /dev/null
> > +++ b/engine/tests/Benchmark.dbench/reference.json
> > @@ -0,0 +1,18 @@
> > +{
> > +    "test_sets":[
> > +        {
> > +            "name":"default",
> > +            "test_cases":[
> > +                {
> > +                    "name":"dbench",
> > +                    "measurements":[
> > +                        {
> > +                            "name":"Throughput",
> > +                            "unit":"MB/s"
> > +                        }
> > +                    ]
> > +                }
> > +            ]
> > +        }
> > +    ]
> > +}
> > diff --git a/engine/tests/Benchmark.dbench/reference.log
> > b/engine/tests/Benchmark.dbench/reference.log
> > deleted file mode 100644
> > index 4b056b0..0000000
> > --- a/engine/tests/Benchmark.dbench/reference.log
> > +++ /dev/null
> > @@ -1,3 +0,0 @@
> > -# Automatically generated on fuego by thresholds.awk based on [628, 670]
> > build range
> > -[Throughput|ge]
> > -0
> > --
> > 2.7.4
> >
> >
> > _______________________________________________
> > Fuego mailing list
> > Fuego at lists.linuxfoundation.org
> > https://lists.linuxfoundation.org/mailman/listinfo/fuego