[lsb-discuss] testing and more
tytso at mit.edu
Fri Dec 22 17:01:10 PST 2006
On Fri, Dec 22, 2006 at 11:40:40AM -0500, Robert Schweikert wrote:
> Yes, but in both cases the app wil crash, thus the first assumption of
> any application developer has to be that the problem lies with them. I
> am not arguing that everything has to be defined, that's impossible.
> Undefined behavior is just that and this has been present for ever. If a
> developer uses an interface in a manner which is said to be undefined
> and then depends on this behavior the developer is certainly not writing
> portable code. (I can without much effort think of a few words I would
> call such a developer)
Actually the first assumption of the users and the netscape developers
was that the application worked just fine with the older version of
glibc, and but when the system was upgraded to the newer glibc (the
one that didn't allow you to fclose() a file handle twice), the
application broke. Since the last thing that changed on the system
was the new glibc version, of *course* the application developers and
the users blamed glibc, even though it was a bug in the Netscape
browser. True story! And if you think about it, it makes perfect sense.
> Of course the tests have to be reviewed and the test submitter (by way
> of the test) has to prove that the bug is not in the application.
> Frankly I don't care if the app crashes. When the application crashes I
> can fix it, when the X-server goes I just ruined my customers work day,
> not very good advertisement for our product.
I guess I have the exact opposite set of priorities. If the X server
crashes, that's something which needs to be submitted to the upstream
developers as part of a bug report, and it will actually probably get
fixed pretty quickly. Once it gets fixed, it probably belongs in the
application's regression test suite, so that if there is ever a
regression it will get caught quickly. In the case of a userspace
program that can cause a kernel crash, the right place for that is the
LTP (Linux Testing Project). And, typically once the problem is
fixed, it generally stays fixed (especially if the test is in merged
into some regression test suite). So the getting the test into the
appropriate regression test suite is far more important than whether
or not it is included in the LSB certification test suite.
My much bigger concern is for application programming mistakes that
lead to a loss of portability. Because these are problems that don't
necessarily go away, since the number of X windows applications vastly
outnumber the X server implementation. So if even if we fix a bug
where one application from one vendor is making programming mistakes
such the application works just fine with Metacity, but crashes and
burns under Sawfish or Kwin, some other application programmer from
some other vendor might come along make the same mistake next month.
So how we help application programmers not make these sorts of
portability mistakes is to me a much more important long-term
strategic question, as opposed to an X server bug that will probably
be fixed within hours or days of bringing it to the attention of the
correct X.org server developer.
More information about the lsb-discuss