[Ksummit-discuss] [MAINTAINERS SUMMIT] Bug-introducing patches

[Laurent Pinchart]

On Saturday, 8 September 2018 00:06:33 EEST Mauro Carvalho Chehab wrote:
> Em Fri, 7 Sep 2018 11:13:20 +0200 Daniel Vetter escreveu:
> > On Fri, Sep 7, 2018 at 6:27 AM, Theodore Y. Ts'o <tytso at mit.edu> wrote:
> > > On Fri, Sep 07, 2018 at 01:49:31AM +0000, Sasha Levin via
> > > Ksummit-discuss wrote:
> > > 
> > > There actually is a perverse incentive to having all of the test
> > > 'bots, which is that I suspect some people have come to rely on it to
> > > catch problems.  I generally run a full set of regression tests before
> > > I push an update to git.kernel.org (it only takes about 2 hours, and
> > > 12 VM's :-); and by the time we get to the late -rc's I *always* will
> > > do a full regression test.
> > 
> > This is what imo a well-run subsystem should sound like from a testing
> > pov. All the subsystem specific testing should be done before merging.
> > Post-merge is only for integration testing and catching the long-tail
> > issues that need months/years of machine time to surface.
> > 
> > Of course this is much harder for anything that needs physical
> > hardware, but even for driver subsystems there's lots you can do with
> > test-drivers, selftests and a pile of emulation, to at least catch
> > bugs in generic code. And for reasonably sized teams like drm/i915
> > building a proper CI is a very obvious investement that will pay off.
> 
> IMHO, CI would do even a better job for smaller teams, as they won't
> have much resources for testing, but the problem here is that those
> teams probably lack resources and money to invest on a physical hardware
> to setup a CI infra and to buy the myriad of different hardware to
> do regression testing.
> 
> Also, some devices are harder to test: how would you check if a camera
> microphone is working? How to check if the camera captured images
> are ok?

The same way you would check the display output. Cameras can be pointed at 
known scenes with controlled lightning. TV capture cards can be fed a known 
signal. Even for microphone testing we could put the camera in a sound-proof 
enclosure, with an audio source. Solutions exist, whether we have the budget 
to implement them is the real question.

-- 
Regards,

Laurent Pinchart