[Ksummit-discuss] [CORE TOPIC] Testing

[Masami Hiramatsu]

On 2015/07/08 3:47, Steven Rostedt wrote:
> On Tue, 7 Jul 2015 14:14:11 +0100
> Mark Brown <broonie at sirena.org.uk> wrote:
> 
>> On Tue, Jul 07, 2015 at 04:02:13PM +0300, Alexey Dobriyan wrote:
>>> On Tue, Jul 7, 2015 at 12:24 PM, Mark Brown <broonie at kernel.org> wrote:
>>
>>>>  - Should we start carrying config fragments upstream designed to
>>>>    support testing, things like the distro config fragments that keep
>>>>    getting discussed are one example here but there's other things like
>>>>    collections of debug options we could be looking at.
>>
>>> This will gravitate everyone to running the same config which is the opposite
>>> of what you want.
>>
>> Perhaps, perhaps not - it's not an unequivocal thing either way.  The
>> more barriers there are to enabling things the more likely it is that
>> people just won't bother in the first place (or that they'll run into
>> somme problem and give up before they get things working) and it's not
>> clear that having to figure these things out is always a good use of
>> people's time.
> 
> The testing/selftests tests should have three results: PASS, FAIL,
> UNSUPPORTED. The UNSUPPORTED is what should be returned if the kernel
> configuration doesn't have the needed features configured. For example,
> if you run the ftrace selftests without function tracing enabled, all
> the tests that test the function tracer return UNSUPPORTED.

This may be an off-topic, but I'd like to ask the selftest for tools.
Currently tools/testing/selftests tests the kernel itself, but
there are many tools under tools/, like perf too.

Those are not configured by the kconfig, but selftests are also needed
for tools. I have a runtests script which is just a bit modified
ftracetest for perf-probe. I'd like to integrate it to selftests
but I'm not sure that is a scope of kselftests.

> Perhaps we should have a central location that each test needs to add
> the required configuration for it to be properly tested. Then if users
> want to test various subsystems, they would look in this location for
> the proper configs (be it a directory that has files of the tests they
> represent, and contain the configs needed). Then there should be no
> real barrier for people to run these tests.

/proc/kconfig[.gz]? I think we can add a list of required kconfigs
for each testcase and indicate it. Moreover, we can include it as
a part of kconfig and introduce CONFIG_KSELFTEST to enable those
configs :)

Thank you,

> 
> Of course if the test requires certain hardware, or a file system, then
> that should be properly documented.
> 
> -- Steve
> _______________________________________________
> Ksummit-discuss mailing list
> Ksummit-discuss at lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/ksummit-discuss
> 


-- 
Masami HIRAMATSU
Linux Technology Research Center, System Productivity Research Dept.
Center for Technology Innovation - Systems Engineering
Hitachi, Ltd., Research & Development Group
E-mail: masami.hiramatsu.pt at hitachi.com