[Ksummit-discuss] [CORE TOPIC] stable workflow

[Mark Brown]

On Mon, Aug 01, 2016 at 04:35:58PM +0300, Laurent Pinchart wrote:
> On Friday 29 Jul 2016 16:12:47 Mark Brown wrote:
> > On Fri, Jul 29, 2016 at 11:59:47AM +0300, Laurent Pinchart wrote:

> > > Another limitation of kselftest is the lack of standardization for logging
> > > and status reporting. This would be needed to interpret the test output
> > > in a consistent way and generate reports. Regardless of whether we extend
> > > kselftest to cover device drivers this would in my opinion be worth
> > > fixing.

> > I thought that was supposed to be logging via stdout/stderr and the
> > return code for the result.

> Yes, but that's a bit limited. For instance we have no way to differentiate a 
> test that failed from a test that can't be run due to a missing dependency as 
> the value of the error code isn't standardized.

I actually went and looked to see where we're at now - there are
standard exit codes for this in kselftest.h following a discussion a few
years ago which are getting some use (via helper functions also in
there).  We've got pass/fail, expected pass/fail and skip.

> Standardizing format for the success or failure messages could also improve 
> consistency. I'm not advocating (at least for now) for any specific format, 
> but outputting messages in a standardized format that can easily be consumed 
> by test runners (e.g. TAP [0], but that's just an example) could be 
> beneficial.

There's some stuff for summary lines in there but yes, this could use
some work.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 473 bytes
Desc: not available
URL: <http://lists.linuxfoundation.org/pipermail/ksummit-discuss/attachments/20160801/387af797/attachment.sig>