[lsb-discuss] testing and more
tytso at mit.edu
Thu Dec 21 21:25:14 PST 2006
On Wed, Dec 20, 2006 at 01:16:39PM -0500, Robert Schweikert wrote:
> Lets take the X.org bug as an example for this one. Since this bug
> effects a large number of people someone is going to come up with a
> hopefully relatively simple test case. Since the bug is triggered by
> using only LSB interfaces the test case should become part of the LSB
> test suite (I understand that we may not be able to add every test for
> every LSB component, but lets just continue this train of thought for
Actually, that may not follow. Just because a bug is triggered only
using LSB interfaces doesn't necessarily mean that the test case is
appropriate. The LSB doesn't necessarily specify all behaviors of a
particular interface, and it may be that the application may be buggy
because it makes assumption about the implementation that may not be
true across all distro's, or may change over time.
Or it could be argued that the specification is buggy, because it is
silent where it needs to say more. So the test case may be right, but
it might not be something we can accept without also making spec
This brings up a related problem that has been gnawing at me for a
while now, so I'll use this message as an excuse to write my
One of the things which worries me as we start standardizing more
libraries in order to support X/Desktop applications is that simply
worrying about interface spceifications may not be enough any more.
For example, there is a particular application (name omitted to
protect the guilty, but some folks may be able to guess) which I am
forced to use every day at work. Unfortunately, this application is
incredibly buggy, since the developers only wrote and tested it on
RHEL 3(!), and it makes assumptions about how the window manager
works, such that it either crashes or runs at varying levels of
usefulness (from minor annoyances to not working at all) if you use
any window manager other than metacity. It also makes various
assumptions about how the CUPS libraries work such that if you run it
on anything other than RHEL 3 and your printer configuration changes,
it will crash on you. The amazing thing is that people actually pay
serious amounts of money for this propietary program....
Right now the only way I can get it to work on a modern distribution
is because a colleague was able to figure out how to write a set of
LD_PRELOAD hacks to intercept the application's buggy use of library
functions (and I've learned to live within the constraints of metacity
and dropped my custom sawfish customizations I had written in Lisp).
Most unfortunately, I don't have source access so I can't fix the bugs
I can easily imagine such a horribly buggy, non-portable application,
obeying every line of the LSB specification, but still not working on
anything but the original distribution it was written to run upon.
The problem is that life is a lot more complicated than just
interfaces. Even if you are just talking about the base X libraries,
the need to obey such messaging protocols such as the ICCCM means that
just because you use the LSB-blessed interfaces is no guarantee of
portability if you screw up your ICCCM compliance. And GNOME and hal
makes this even more complicated, because their rules might not
necessarily be well documented yet.
How do we fix this? I'm not sure, but it's something for us to think
about. As we start standardizing libraries that have more stateful,
object-oriented, or messaging-based interfaces, it seems pretty clear
to me that simply standardizing C/C++ function signatures is not going
to be enough.
And even once we create specifications, writing test cases to make
sure that the libraries are and the applications under a certification
test are correctly behaving is yet another challenge....
More information about the lsb-discuss