[Ksummit-discuss] [CORE TOPIC] kernel unit testing

[Jan Kara]

On Mon 11-07-16 12:58:50, Dan Williams wrote:
> * What does "we never regress userspace" mean when we have unit tests
> that are tightly coupled to the kernel?
> For example, there have been occasions where I have "regressed" a test
> case to improve kernel behavior with the knowledge that no real
> (non-test) application was dependent on the old behavior.

It's always a judgement call. Linus has always stated that the rule only
holds for real users using real applications. So if you are confident that
it is only the test that regresses, then that is fine.

> * Are tests only for developers, or should an end consumer of the
> kernel expect to be able to run the tests and baseline their kernel?
> The unit tests for libnvdimm [2] are meant to run against latest
> mainline. There's some support for checking the kernel version, but I
> don't promise that tests for new kernel functionality will be skipped
> on older kernels without that specific enabling. I'd revisit that
> stance if -stable tree maintainers or distros were looking to run the
> tests.

Well, in the end it is your call but my experience with xfstests shows that
it is good to detect the case that the functionality is not supported by
the kernel and fail gracefully. Because especially with enterprise
distributions you get a strange mix of kernel and userspace and it is *very*
useful to be able to run testsuites there when testing backports etc. It is
a pita to figure out whether the test failed because of missing
functionality of because of a bug in your backport...

								Honza
-- 
Jan Kara <jack at suse.com>
SUSE Labs, CR