[agl-discussions] AGL image stability, stress and functionality tests proposal

[Bocklage, Jens]

Ok. Sounds like I can/should start NOW.
Who can create a tag on the AGL repos? I guess it would be a good idea to have Qt5 integrated before tagging.
There is a patch from Tanikawa-san that applies the needed changes. Can we somehow get speed into that?

Jens

From: Streif, Rudolf [mailto:rstreif at jaguarlandrover.com]
Sent: Mittwoch, 16. September 2015 18:07
To: Bocklage, Jens; automotive-discussions at lists.linuxfoundation.org
Subject: Re: [agl-discussions] AGL image stability, stress and functionality tests proposal

Thanks, Jens, for your response and making me aware of that I missed to copy the ML on my first response.



A couple of suggestions and rough ideas:

  *   I think some of the testing has to be carried out on actual reference hardware, other portions we may be able to do using a virtual machine or emulator. It may be worthwhile adding a field to the test case descriptions indicating the test target options.
My idea is to test it on the porter HW, since this is available for us. Do you think we also should test the qemu-version? The “test target options” field is a good idea!

The Porter is a good board to test on. I think we might want to add another board, one that is more ubiquitously available. I don't even know how to get my hands on a Porter (the board not the beer :). I do have Wandboard and MinnowBoard Max though.

In general it would be good if the AGL distro runs on QEMU too. It is an easy way for folks to test it after they built it for the first time. Then you can move on to hardware. It's how I do and like it. Some automated tests could also be more easily carried out on an emulated system.


  *   What are your thoughts on actually performing the tests? Would that be a person or automated tests? I think for most part the latter is probably preferable. Have you looked into automated testing tools?
As a start, the tests are executed manually. Some may be executed via a script. But the goal should be to have it automated (maybe as part of the CI as a Jenkins-job?). I have not looked inte automated testing tools so far.

I think that's a reasonable start. The test cases to some extend also drive the choice of test tools. Scripting is a good thing to verify general system setup etc. For UI testing other tools that simulate user interaction are necessary. Jenkins CI is good to integrate and orchestrate them.


  *   Test report and verification of the results: ideally all expected results can be verified using automated tools and the actual result automatically collected into test reports. Is that in line what you have in mind?
Yes.

I think we are on the right path.

Cheers,
Rudi

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxfoundation.org/pipermail/automotive-discussions/attachments/20150917/e82966d7/attachment-0001.html>