Submitted for your approval: White paper draft

[Nicholas Petreley]

Folks,

Out from the twilight zone and into your hands, here is a white paper describing LSB, some of the reasoning behind our standards and LSB compliance benefits to customers, developers, ISVs and Linux suppliers.  The idea is to publish this in a few places and make it generally available in order to educate press and analysts as to what LSB is really all about, and get them speculating on how wonderful it will be for everyone who touches it. ;-)

Please comment, suggest additions, deletions, modifictions, the titles of any good books you've read lately, whatever.  

=======================================

The goal of Linux Standard Base (LSB) is to develop and promote standards that will increase compatibility among Linux offerings and enable software applications to run on any compliant Linux system. In addition, LSB will help coordinate efforts to recruit software vendors to port and write products for Linux.  

This document addresses the following: 

Why customers need LSB
Why developers and ISVs need LSB
Why Linux distributors needs LSB

Customers need to be able to purchase the Linux distribution that best suits their needs without fear that their applications will have compatibility problems on that distribution.  LSB assures a customer that their choice of application has been tested against LSB standards and should run properly on any LSB compliant distribution.  

Developers and independent software vendors (ISV) alike need LSB because it is in their best interest to count on their applications to run on as many Linux platforms as possible.  Developers reach the broadest possible audience without having to form political alliances with every possible Linux provider.  

Linux distributors need LSB because it provides a clear framework around which they can add unique value to Linux without breaking applications. 

SUBHEAD: Solutions-based standards

The Linux Standard Base will include a written specification, a test suite to measure compliance, and a sample implementation of an LSB-compliant base distribution of Linux as a starting point for others to use if they wish.  The sample implementation is also a useful development and testing tool for developers and ISVs.  

The scope of LSB standards will include the monitoring and standardization of: 

The most widely used shared libraries
System commands
Filesystem hierarchy
System initialization process and scripts
POSIX.1 compliance plus glibc extensions
Sockets
X11 (minimal)
Basic software installation
Object file format


The goal of LSB is to assure cross-distribution and backward compatibility of Linux applications without impeding innovation in Linux.   Shared libraries are at the root of many application compatibility issues, especially when libraries are not subjected to strict version control or when application writers don't know which version of a library to use. 

Most operating systems rely upon shared libraries to provide applications with a set of standard functions and utilities without wasting storage space.  Linux, for example, usually includes essential and commonly used libraries such as glibc, pthreads, libm, Xt, ncurses, among many others.  Applications which are compiled with a given version of a shared library will expect to find precisely the version they need at run time.  

While it is possible for these applications to run with a later version of a library, this backward compatibility cannot always be guaranteed.  For example, there are known compatibility issues between similar libraries such as glibc 2.0, glibc 2.1, and glibc 2.2.  If an application has been compiled to run properly with glibc 2.1, it is unlikely that the application will run with an earlier version, and there is no guarantee it will run properly with a later one.  

One way around this problem is to include multiple versions of libraries within a Linux distribution and allow applications to select those they were built to use.  While this sometimes works, it isn't always practical because the space used can become excessive as you add library versions, thus undoing the value of having shared libraries to begin with.  

As a result, there are often scenarios where a system will end up with what appear to be several versions of a library, but which are really several links that point to a single file. While this is a common tactic used to compensate for the fact that multiple applications may require different minor versions of a given library, it is impractical as an LSB strategy to solve library compatibility.  In the first place, it simply isn't reliable enough to be an acceptable solution.  It also does nothing to prevent an installation from overwriting libraries with newer versions that may break backward compatibility. 

LSB therefore defines a minimal set of libraries as part of the base distribution.  LSB gives approved libraries unique names identifying them as LSB-compliant libraries.  For example, the LSB version of libcrypt might be libcrypt.lsb.1. This naming convention establishes a known set of library definitions upon which application developers and vendors can rely.  

The separate naming convention also prevents installation programs from overwriting LSB-compliant libraries with new versions of the same library.  If, for instance, a Linux provider included a version of libcrypt that provides new features added since libcrypt.lsb.1, it would be dropped into the library directory using a non-LSB naming convention such as libcrypt.so.1.2.  This file may replace libcrypt.so.1.1, but it would never replace libcrypt.lsb.1.  This makes the new features available to applications that need them without interfering with applications that need the LSB libraries.  LSB deliberately allows these new libraries and LSB-compliant libraries to coexist to ensure compatibility without impeding progress.  

It is possible for an operating system vendor to conceivably keep competitors off-balance by replacing core libraries with each new version or incremental update to its operating system.  Fortunately, the Open Source nature of Linux already makes it nearly impossible for any single Linux provider to control standards in this manner, because the latest versions of core Linux libraries are freely available regardless of the Linux distribution you use.  But LSB adds one more level of insurance, because customers are assured that an LSB compliant Linux offering will not break LSB compliant applications even if it includes the latest libraries.  

LSB also considers certain commands (such as cat, sleep, more, man, sed, nice, tail, etc.) as mandatory for a base distribution. LSB is similar but not quite identical to The Open Group's Single UNIX Specification Version 2 (a.k.a. UNIX 98) specification with respect to its stated requirements.  Linux and its applications rely heavily on shell scripts to do things like install, remove, initialize, and shut down software.  Certain commands must be present so that existing scripts can run without errors, and so that developers can continue to write shell scripts without fear that the basic tools they choose will be missing.  

It is not only important for an LSB compliant to contain a minimum set of commands, scripts, programs, and users need to know where to find them.  LSB has adopted the Filesystem Hierarchy Standard (FHS) to accomplish this goal, among others.  FHS dictates how files, libraries and commands are arranged in an LSB compliant system.  Visit  http://www.pathname.com/fhs for a complete description of this standard and what it covers. 

LSB recognizes that some LSB compliant Linux applications can and will be started when the Linux system boots.  For this reason, LSB defines a standard procedure for the placement and contents of initialization files and scripts.  This standard allows a database vendor to install the proper database initialization scripts with the assurance that other initial services upon which it may depend are already started.  The standard also prevents things like circular dependencies, which would cause system initialization to go into a never-ending loop.  

LSB compliant systems support the POSIX.1-1990 standard plus glibc extensions. This provides developers with a known set of functions and a reference upon which they can rely when developing applications.  

LSB specifies the socket interface for access to the Internet.  More and more applications are making use of the network, and as such, the LSB enforces the existing standard most applications already use for Internet access.  

While not all applications use a graphical user interface, a large enough percentage of them do that this area can't be ignored.   LSB defines the lowest layers of X11, which should help reduce application size by not requiring them to link these libraries statically.  This continues to allow applications the freedom to choose from among the many popular higher layers, or perhaps to display X11 applications on another computer (including one that is not LSB compliant or even runs some other operating system than Linux).   

Executable and Linking Format (ELF) is the de-facto standard executable object file format used with Linux.   LSB reaffirms the use of this standard and tests for it.  The ELF specification is available in pdf format at the URL http://www.linuxbase.org/spec/refspecs/elf.pdf. 

SUBHEAD: Less is more

What isn't covered by the LSB is equally as important as what is covered.  Linux customers, ISVs, developers and Linux distribution vendors will not only benefit by the standardization issues that the LSB addresses, but in some cases they reap even more benefits because there are things LSB specifically will not address.  

It is important to understand that LSB focuses only on the elements of the Linux operating system that affect application compatibility.  Consider this to be a layer between the Linux kernel and features of a distribution that are not required by applications, such as Linux installation programs or the third-party applications themselves.  

The LSB defines a line below which all things should remain in the known state necessary to ensure compatibility for applications.  But by implication, above this line almost anything goes.  By giving Linux distributors a clear boundary below which nothing should change, ISVs and developers also get a clearly defined boundary above which they may freely differentiate their products without creating compatibility problems.   

As long as a Linux distributor respects the boundaries of LSB-compliance, it is possible to add proprietary software that does not subvert the benefits of Open Source to the customer.  Once a proprietary installation program is finished installing Linux, for example, it essentially disappears and no longer  affects that distribution's compatibility with LSB-compliant applications.  

On the other hand, another distribution vendor may see fit to compete strictly on the grounds that all of its code is Open Source with the intention of attracting customers who want the source code for everything, including programs that are peripheral to LSB compliance.  

LSB also deliberately avoids dictating how system administration works on Linux.  For example, LSB will not require a strict set of default security policies for users, groups and file systems.  LSB compliance requires that applications will continue to function if the system administrator modifies the default security settings (Within reasonable limits, of course.  Obviously a hapless administrator can damage any system.  LSB does not - and could not - prevent such things.)  

SUBHEAD: The customer wins

In the end, there are many potential benefits to adopting LSB standards.  Well-defined standards provide non-commercial and commercial developers alike with reliable guidelines for producing good code.  Broad LSB compliance means applications will run on every Linux box instead of being designed for a smaller segment of the Linux market.  LSB will enable a larger market for every provider of Linux because it encourages more Linux suppliers, value-add resellers, developers and independent software vendors to support Linux.   There remains a strong incentive for everyone to innovate outside of the boundaries of what determines application compatibility without posing a threat to the inestimable value of the Open Source nature of Linux itself.  

[VERY brief wrap up of standard information, such as...]

Current Members include Caldera Inc, Corel Corporation, The Debian Project, Enhanced Software Technologies, Inc., IBM, LinuxCare, Linux for PowerPC, MandrakeSoft, Metro Link, Inc., The Open Group, Turbolinux Inc., Red Hat Inc., Software in the Public Interest, Inc., SuSE. GmbH, VA Linux, WGS Inc., and SGI.  



-Nick

-- 
**********************************************************
Nicholas Petreley                   LinuxWorld - InfoWorld
nicholas at petreley.com - http://www.petreley.com - Eph 6:12
**********************************************************
.