[Fuego] 答复: Problem of adding a benchmark result table

[Li, Xiaoming]

Many thanks for the information.

I could not wait to take a dive into them


-----邮件原件-----
发件人: Tim.Bird at sony.com [mailto:Tim.Bird at sony.com] 
发送时间: 2018年8月29日 1:45
收件人: Li, Xiaoming/李 霄鸣; fuego at lists.linuxfoundation.org
抄送: daniel.sangorrin at toshiba.co.jp
主题: RE: [Fuego] Problem of adding a benchmark result table

A few more comments inline below...

> -----Original Message-----
> From: Li, Xiaoming
...
> My  reply is below.
> 
> > -----Original Message-----
> > From: Tim.Bird at sony.com
> > > -----Original Message-----
> > > From: Li, Xiaoming
> > >
> > > Hi Tim,
> > >
> > >
> > >
> > > As I am urge of implement this function, could u give me some 
> > > hints or advice on questions below?
> >
> > You should look at:
> > http://fuegotest.org/wiki/Jenkins_Visualization
> > It has some information about the charting features of Fuego, and 
> > the integration with Jenkins.
> 
> Great, I found much valuable info there.
> 
> >
> > >
> > >
> > >
> > > 1)If modiying the mod.js to generate a single benchmark report, 
> > > does it need the batch_id arg? or parse the report file with own algorithm?
> >
> > Currently, Fuego does not have a batch_id implemented.  It does not
> appear
> > in run.json, or in flat_plot_data.txt, or flot_chart-data.json.  
> > Although it is mentioned in the ftc command-line help for --where, 
> > it isn't actually implemented yet.
> > (Note that if run.json *had* a batch_id field, the code in ftc that 
> > handled -- where clauses should function properly for filtering 
> > this.)
> >
> > prepare_chart_data.py parses the run.json data with its own algorithm.
> >
> > >
> > > 2)My plan is to add a chart on top of test suite's flot graph, 
> > > with a menu of selecting build no. (sorry, I am not an expert of 
> > > web design)
> > This sounds useful, but might be complicated.
> > Does this 'chart on top of ...the flog graph' contain data, or is it 
> > just a list of checkboxes for selecting the data to include in the graph?
> >
> > Just to clarify the terminology, I use the following (hopefully consistently):
> > chart = visualization item, which can be a table or a plot (graph)
> >
> > There are three types of charts:
> > 1) measure_plot = a 'flot' plot of benchmark measures
> > 2) testcase_table = an HTML table of testcase results
> > 3) testcase_summary_table = an HTML table of summary counts per
> testset
> >
> > I wanted for the system (prepare_chart_data.py and mod.js) to 
> > support being able to show multiple charts for the same test, and 
> > being able to support pre-filtering (by prepare_chart_data.py) and 
> > visualization-time filtering (by mod.js using flot capabilities to 
> > turn items on and off in the Jenkins user interface).
> >
> > I'm not sure, however, of the status of this code.  It looks like 
> > mod.js can read multiple charts from the flot_chart_data.json file, 
> > but that prepare_chart_data.py only writes one chart (or one kind of 
> > chart) to that file.
> 
> My head is messed up by their complex relation.
> Maybe I'd better  read the code fully before starting new work.

It would probably help to look at the files before and after a run of a simple test (with multiple test cases) or benchmark.

The key files are:
 testlog.txt - contains the un-parsed standard output from the test, generated by the test program on the DUT
   - one per run
 run.json - contains the data for a single test run, generated by Fuego parser from testlog.txt
   - one per run
 chart_config.json - contains configuration for the charts for a test - created by test author and is in test home directory
   - one per test
 flat_plot_data.txt - contains a set of lines for all runs for a particular test, generated by prepare_chart_data.py
   - one per test, one line per testcase  flot_chart_data.json - contains data for display by mod.js - either HTML tables or data needed by flot Javascript graphing system
  - one per test, one structure per chart
 
> 
> >
> > >
> > >
> > > 3)Do u have any schedule of re-investigating the fuego chart（or
> > > graph,table...） related mechanism?
> > I don't have a schedule.  I might be able to take a look at it this 
> > week, but I
> am
> > WAY behind in my patch review for new tests and test fixes.  Don't 
> > get me wrong - I am grateful for all the work by Fujitsu engineers, 
> > but I need to
> catch
> > up on about 20 to 30 patches.  It will take me probably a few days 
> > at least to get to this.
> 
> Please forgive my rudeness,  which is probably due to my poor 
> English(or differences between culture ).
> It is not my intention.

I did not interpret anything you said as rude.  I was just trying to explain why I might not be able to work on this for a while.
I am grateful for Fujitsu's great contributions to Fuego.
 -- Tim