[Lsb-CommonPackaging] Re: extension of lsb packages

Anthony W. Youngman Anthony.Youngman at ECA-International.com
Wed Mar 6 07:31:46 PST 2002

:-( Lookout seems to be mangling my inlines... Apologies if it's hard to

-----Original Message-----
From: Theodore Tso [mailto:tytso at mit.edu] 
Sent: 06 March 2002 14:37
To: Anthony W. Youngman
Cc: 'msw at redhat.com'; lsb-discuss at lists.linuxbase.org;
lsb-taskforce1 at lists.sourceforge.net; George Kraft IV
Subject: Re: [Lsb-CommonPackaging] Re: extension of lsb packages

On Wed, Mar 06, 2002 at 08:10:19AM -0000, Anthony W. Youngman wrote:
> Which is why I'm trying to get a new standard that says "here's an 
> api. With this, ANY packaging program can talk to ANY distro package 
> database and query it".
> Bingo. All packaging problems solved. You can run any installer on top 
> of any distro and check dependencies, log the fact that you've 
> installed stuff, etc etc. The trouble with current install mechanisms 
> is that they are installer, DBMS, packager, and god knows what else 
> all stuffed into one bundle. If we separate the separate functions 
> into separate tools like all good linusers do, then the problem gets a 
> lot simpler.

If you assume that the packaging system is responsible for installing
packages, then it isn't that hard.  You just simply have a couple of
operations, really:

	1)  /usr/lsb/lib/install_package <package_file>
	2)  /usr/lsb/lib/remove_package package_name [version]
	3)  /usr/lsb/lib/query_package  package_name [version]

Of course, then the only reason why you need a fancy installer program is to
allow the user to choose which sub-packages the user wishes.

Well, I was thinking possibly along those lines...

If you're thinking more in the Windows model, where each vendor has their
own propietary setup.exe program which is responsible for copying files in,
then that becomes a horse of a very different color. You need to realize
that (a) the packaging system keeps track of which file is owned by which
package, and often keeps a checksum information about files, so it can
determine if a package has been corrupted, 

So when the installer tells the package database what it's installed, it has
to include a list of files ... fine.

(b) it's kinda stupid to have each vendor replicating the code to copy a
file into a standard location, and 

It's also kinda stupid to have loads of static and/or duplicate code where
different lsb packages just happen to share functionality that isn't defined
in the lsb spec. It would be nice for eg Kylix to say it had installed all
the dev libraries, so when the user installs an app written by someone else,
the installer realises it doesn't need to install duplicate copies of all
the runtimes... maybe several times over if I install several Kylix-based
packages from several different sources.

(c) the API between the installer program and the packaging system gets
*very* complicated, because of the need of the installer to report where
each file is to be installed first.

It also doesn't solve any of the hard problems of packaging, which include:

1)  Packaging namespace.  Each distribution assumes that it, and only it,
controls the namespace of packages.  So they will freely create packages
without worrying about collisions.

Already dealt with. We've dealt with it in lsb by creating an lsb namespace.
The lsb imposes a domain-name-style namespace. Don't comply? You're not

2)  Version numbers.  Each distribution uses a versioning scheme which is
roughly related to the original upstream author's versioning scheme, but
there are both "epoch" numbers which are more significant than the base
version, as well as "release" numbers which are less significant than the
base version.

Again, dealt with. We may need exceptions, but in general we ought just to
be able to use the original author's numbering scheme. What happens if there
are functionality-damaging bugs and the original author doesn't fix it I
don't know, but that's hopefully an unlikely occurrence.

3) Incompatible event models for installation and deinstallation and upgrade
scripts.  The pre- and post- install scripts can be hand-waved away if you
assume the propietary, hand-written installation program for each package.
(Although this clearly wouldn't scale for the several hundred packages in a
base distribution, but we'll ignore that point for now).  However, how and
when the scripts which get run when the package is deinstalled or upgraded
*do* need to be standardized, since the installation program won't be
running at that point.

How does lsb do it now? Same problem, and same solution!

4) Detailed configuration information for each base package to be provided
by the distribution.  The FHS provides general guidance to the
distributions, but there are all sorts of details about how Apache is to be
configured, or what the precise directory name is used in
/var/lib, which do differ from distribution to distribution.   

As above. How do we do it now? If my lsb app needs Apache, then my spec
suffers exactly the same difficulties as the lsb suffers now. And it will
presumably be able to take advantage of exactly the same solution.

5) Knowledge of how to modify system configuration.  If a package needs to
add a port to /etc/services, how should it do so such that it will survive
an update of the base distibution?  How about /etc/inetd.conf?  Etc.

And again. As above.

All of these combine to make the problem of dependencies very, very, hard.
If my program needs to be able to send mail, what package (or virtual
package) do I need to list in my dependencies to guarantee that exim or
postfix or sendmail will be installed?


What package do I need to depend upon to guarantee that Apache will be
installed.  And if my package is providing an Apache plug-in module, where
has the distribution decided to place the standard Apache plug-ins?  For
that matter, where has the distribution decided to place the Apache
configuration files?  Or log files?


And if my package requires at least Apache 1.3.12, how do I state that?
Different distribution may have different epoch numbers, and in some cases,
different ways of encoding the version number if the upstream author hasn't
been careful about releasing software with consistent version numbers.

With an lsb-defined namespace, we simply say Apache >= 1.3.12. Simple!

So to say, "all we need to do is to specify an API", and like pixie dust,
expect that all of the hard problems of packaging will just magically
disappear, is a bit naive.  

Yes, I know it is incredibly naive. But I fail to see how, using my
solution, it's going to be *harder* to solve than using the current
mechanism. The solutions we come up with here will carry over.

The LSB 1.0 has a partial solution to these problems, but we do so by
significantly constraining what an LSB-compliant package can do.  An
LSB-complaint distribution can only depend on a single package, named "lsb".
We restrict what sort of things the package is allowed to assume (because
they're different on various distributions).  We require long package names,
such as "lsb-quicken.com-turbotax", which are not universally popular.

And I've simply sought ways to *extend* that, with the intention of
increasing flexibility. Just like with lsb 1 we restrict heavily what the
package is allowed to assume, I've done the same, with my "necessary and
sufficient" set (which pretty much means "if you require more than current
lsb, then you're broken"). But it also allows for the addition of a lot of
*OPTIONAL* extras, which allows packagers to innovate, try things out, and
then ask for them to be added to the standard.

> Of course, the fly in the ointment is this becomes a "standard 
> creating" exercise, rather than the current "codifying existing 
> standards" exercise and there'll be horrendous politics. But if you 
> think about it, the proposed approach will work perfectly, even with 
> an ISV who wishes to create a totally proprietary, obscured to the 
> hilt, 100% totally self owned code, setup.exe to run on a 
> lsb-compliant system, AND IT WILL WORK. If we can address and allow 
> for that setup, it then makes it easy for every one else to be far 
> more sharing and co-operative.

Well, ask yourself this.  What parts of a propietary, obscured to the hilt,
hand-rolled installer would the ISV really want to keep secret? And why?
Copying files around?  That's not secret.  The list of what files are
installed?  We can compare the system before and after the installation, and
know what files where installed/modified.

That comment was not meant seriously. It's just a genuine worst-case
scenario, and as I said, if we can deal with that then it'll be dead easy to
deal with developers working in friendly co-opetition.

The only thing which really needs to be kept secret/propietary is the
initial configuration of the package, and arguably that shouldn't be part of
the installation process in the first place.  You *want* to separate that
out, and perhaps the most you should do is to be able to call the initial
configuration tool from the post-installation script.  

The bottom line is that I don't think there's any real value in having a
windows-style SETUP.EXE custom installer.  People may be accustomed to it
because it's what Windows used to do, but even in the newer Windows system,
Microsoft has actually been moving to the packaging model used by Linux
distribution systems, where the vendor provides a package, and the packaging
system takes care of installing it. Indeed, with Windows 2000, Microsoft was
boasting about how for the first time, they can verify whether or not a
package has been corrupted or not --- of course, they conveniently failed to
mention that Linux and its packaging systems (especially RPM) has been able
to do this for years....

						- Ted

Which is why my spec is based on (a) separating the front and back ends of
the installer, (b) abstracting the interface, and (c) trying to spec that
interface such that it's compatible with rpm, apt, and lsb.

Once we've got that, and a mechanism for extending it, then co-opetition
means the whole shebang is going to evolve rapidly.

As I've said before, "monolithic" is NOT the linux way. You wouldn't expect
a database client to directly modify the files on disk, would you? It goes
through a SQL interface or similar and lets a DBMS look after it. Why not
apply *exactly* the same approach to an install tool. After all, the front
end currently DOES modify the back end database, doesn't it? How stupid is

At the end of the day, all the objections you've brought up are either (a)
obvious, and with obvious solutions, or (b) ones the lsb is already falling
over, and we can reuse the same solution.

And all I'm trying to do is abstract the interface between the installer and
the package database, which means we can work out a common subset between
the main packagers, and seek to build on it. At present, I feel all this
debate is having a seriously stifling effect on package management
development, and if we start codifying things the way they are, things are
going to get worse.

I'm quite happy with the way things are going for lsb 1. We need the current
standard. But things need a total rethink for lsb 2. When this came up
before I got the impression Jeff and Wichart grok'd what I was getting at.
Albert most certainly did. And I would not be in the slightest bit surprised
if my ideas didn't get taken up in some form or other. I'm a database
professional. I make my living programming the damn things. And the way the
package databases are managed just feels horribly "inelegant". As you ought
to know, "if it's inelegant, it's wrong!". Maybe I'm wrong, but we are
talking about my personal sphere of expertise...


This transmission is intended for the named recipient only. It may contain
private and confidential information. If this has come to you in error you
must not act on anything disclosed in it, nor must you copy it, modify it,
disseminate it in any way, or show it to anyone. Please e-mail the sender to
inform us of the transmission error or telephone ECA International
immediately and delete the e-mail from your information system.
Telephone numbers for ECA International offices are: Sydney +61 (0)2 9911
7799, Hong Kong + 852 2121 2388, London +44 (0)20 7351 5000 and New York +1
212 582 2333.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.linux-foundation.org/pipermail/lsb-discuss/attachments/20020306/d1ed34a7/attachment.htm 

More information about the lsb-discuss mailing list