[Fuego] [agl-discussions] [RFC]AGL-Fuego output test result change from xml to json
Cai, Song
cais.fnst at cn.fujitsu.com
Wed Apr 12 11:40:13 UTC 2017
Thanks for your kindness.
I can change my schedule according to Tim's one.
--------------------------------------------------
B.R.
Cai Song
--------------------------------------------------
-----Original Message-----
From: Jan-Simon Möller [mailto:jsmoeller at linuxfoundation.org]
Sent: Wednesday, April 12, 2017 7:23 PM
To: automotive-discussions at lists.linuxfoundation.org
Cc: Cai, Song/蔡 嵩 <cais.fnst at cn.fujitsu.com>; Bird, Timothy <Tim.Bird at sony.com>; fuego at lists.linuxfoundation.org
Subject: Re: [agl-discussions] [RFC]AGL-Fuego output test result change from xml to json
Hi Tim / Song,
I can host a gotomeeting bridge if needed. Just ping with the timeslot.
Best,
JS
Am Mittwoch, 12. April 2017, 10:40:09 schrieb Cai, Song:
> Hi Tim:
> Our job about LTP is going to draw to an end.
> So, can you take some time for us to discuss the issues "xml to json
> recently", since both of us have done the work by ourselves before.
> Thursday and Friday this week is available, or other time at your
> convinces. My time zone is UTC +8. Considering the network in China,
> goto meeting maybe be good choice. Looking forward to your reply.
> Thanks.
>
> --------------------------------------------------
> B.R.
> Cai Song
> --------------------------------------------------
>
> -----Original Message-----
> From: Bird, Timothy [mailto:Tim.Bird at sony.com]
> Sent: Tuesday, April 04, 2017 12:21 PM
> To: Cai, Song/蔡 嵩 <cais.fnst at cn.fujitsu.com>;
> automotive-discussions at lists.linuxfoundation.org;
> fuego at lists.linuxfoundation.org Subject: RE: [agl-discussions]
> [RFC]AGL-Fuego output test result change from xml to json
>
> Sounds good. Let me know when you are ready to resume work on this
> part of Fuego. I'm currently doing a lot of integration work with
> Daniel Sangorrin, of Toshiba. Hopefully that will be settled down by
> the end of this week, and we'll have some time to work on the AGL
> integration and unified output and report generation issues. -- Tim
>
> > -----Original Message-----
> > From: automotive-discussions-bounces at lists.linuxfoundation.org
> > [mailto:automotive-discussions-bounces at lists.linuxfoundation.org] On
> > Behalf Of Cai, Song
> > Sent: Friday, March 31, 2017 4:03 AM
> > To: Bird, Timothy <Tim.Bird at sony.com>; automotive-
> > discussions at lists.linuxfoundation.org;
> > fuego at lists.linuxfoundation.org
> > Subject: Re: [agl-discussions] [RFC]AGL-Fuego output test result
> > change from xml to json
> >
> > Hi Tim:
> > Sorry for my late reply.
> > Our team are solving the cases of LTP on 64 bits boards.
> >
> > And we have accepted the AGL community job "integrate AGL-Fuego to
> >
> > Fuego" recently, so we need to keep progress on it after we finish
> > LTP cases.>
> > We can deal with the community job and the json issue at the same
> >
> > time, since it is included in integration.
> >
> > Hope for your understanding.
> > Thanks.
> >
> > B.R
> > CaiSong
> >
> > -----Original Message-----
> > From: Bird, Timothy [mailto:Tim.Bird at sony.com]
> > Sent: Friday, March 24, 2017 3:37 AM
> > To: Cai, Song/蔡 嵩 <cais.fnst at cn.fujitsu.com>; automotive-
> > discussions at lists.linuxfoundation.org;
> > fuego at lists.linuxfoundation.org
> > Subject: RE: [RFC]AFL-Fuego output test result change from xml to
> > json
> >
> > > -----Original Message-----
> > > From: Cai, Song on Wednesday, March 22, 2017 7:19 PM Hello
> > > Jan-Simon, AGL members
> > >
> > > Fuego members consider that it will be more versatile if we output
> > > the test results in json format,
> > >
> > > Since json is widely used, and many tool (e.g. Kernel CI) is also
> > > in json
> >
> > format.
> >
> > > I totally agree with that.
> >
> > I strongly agree with this as well. Our intermediate format for
> > results should be json. By "intermediate", I mean the format used to
> > store results in the system, as opposed to the "display format". We
> > should support multiple display
> > output formats (such as html, xml, pdf, or wiki markup), that can be
> > generated from the json formatted data.
> >
> > > And we plan to change the current result output format from xml to
> > > json, which is scheduled to be finished until late May.
> > >
> > > I would appreciate any comments on this activity.
> >
> > Please note that Fuego already has the capability to put results in
> > json format now.
> > However, the work is scattered in different functions, and could use
> > some better organization and coverage.
> >
> > Here is some information that may be useful in working on this issue:
> > * data for a functional test that is executed by Jenkins is in the
> > file build.xml>
> > * this is converted to a "run.json" file inside the run package
> > if
> >
> > you execute "ftc package-run" for the run
> >
> > * so, ftc has the capability to read the Jenkins build.xml
> > file
> >
> > and produce a Fuego run.json file now
> >
> > * data for a functional test that is executed by 'ftc run-test'
> >
> > creates the run.json file directly, and it is present in the log
> > directory for that run
> >
> > * data for each metric of a benchmark test is placed in a file with
> >
> > the suffix ".res.json"
> >
> > * this is currently only generated when Fuego is executing a
> > batch
> >
> > job (this should be extended to all job execution contexts)
> >
> > * this file is created by parser.py, when the benchmark metrics
> >
> > are collected from the log
> >
> > * additionally, when a batch job is executed, a "logrun" file is
> >
> > created, in JSON format
> >
> > * this file is used to collect the data from the multiple tests
> > in
> >
> > the batch job, so they can be formatted into a PDF by the report
> > generator
> >
> > * There are also some json files used for generating plots using
> > the 'flot'
> >
> > module
> >
> > * there is one file per metric in the log/<testname> directory
> > * there is an 'info.json' file in the log/<testname> directory,
> >
> > which has additional data used for the plot
> >
> > Note that some of these features are currently broken in the 'next'
> > branch, as we work through the integration of the Toshiba and Sony
> > code bases (the Jenkins integration refactoring).
> > I hope to have some of the broken issues fixed by the 1.1 release,
> > and all of them resolved by the 1.2 release.
> >
> > Please see the following pages, which have information on the
> > logrun,
> >
> > res.json, and run.json files:
> > * http://bird.org/fuego/Report_Generator
> > * http://bird.org/fuego/run.json
> >
> > What I'd like to see happen, is for all test result data to be put
> > into the run.json file.
> > This includes data that is now put into res.json, and data that is
> > put into the xlsx files for tests that have lots of sub-results (e.g. LTP).
> >
> > Also, we should make sure that the data and the run.json file are
> > created in all contexts - not just when a test is run from ftc, or
> > when a batch job is executed.
> >
> > Also, it sounds like it would be handy to have the board data in
> > json format. We have python code now that parses the board file
> > (both in ovgen.py and in ftc), and produces python data structures
> > for the board data. It might be possible to reuse this parsing code
> > for your work. In particular, the board data might be getting more
> > complex in the future, as we add the data dictionary feature to
> > Fuego to store dynamic variables for boards. I can trivially add
> > some code to ftc to output the board data in json format, for processing by other tools.
> > Let me know if you would like me to do this.
> >
> > Please let me know what you are planning on doing in this regard,
> > and we can discuss details of the design, to make sure it fits with
> > the long-term vision for Fuego results (and report generation)
> > storage and display.
> >
> > BTW - I saw your slides from the Japan Jamboree presentation. They
> > look good.
> > I'll post a link to them on http://bird.org/fuego/Presentations.
> >
> > Thanks,
> >
> > -- Tim
> >
> > P.S. Note that I've made a slight modification to reports.sh in my 'next'
> > branch, just today, based on this discussion. The function
> > set_testres_file in reports.sh used to only set the GEN_TESTRES_FILE
> > when operating in batch mode. However, I've removed this condition.
> > Tests should now always create a res.json file.
> > See https://bitbucket.org/tbird20d/fuego-> >
> > core/commits/6cd8fe8bde38887b12b1a5b1713877a0ec754a2d
> >
> >
> >
> >
> >
> > _______________________________________________
> > automotive-discussions mailing list
> > automotive-discussions at lists.linuxfoundation.org
> > https://lists.linuxfoundation.org/mailman/listinfo/automotive-discus
> > si
> > ons
>
> _______________________________________________
> automotive-discussions mailing list
> automotive-discussions at lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/automotive-discussi
> ons
More information about the Fuego
mailing list