[lsb-discuss] RPM decision process

[Theodore Tso]

On Wed, Apr 16, 2008 at 03:45:06PM -0400, R P Herrold wrote:
> My opening remark to the comment that 'We agreed at the face- 
> to-face that RPM package version format uplift was not 
> important and might be deferred' was -- Either uplift to RPM 
> package version 4, or just use tarballs [ar, cpio, XML blobs, 
> whatever];  RPM package version 3 is long obsolete, 
> incompletely protects the content, and is subject to a known 
> modification attack which cannot be detected in the way the v 
> 3 signing leade is used.

I think some of the confusion may be based on what goals you think the
packaging effort within the LSB is trying to solve.  For example
cryptographic signatures matter a lot for community based
distributions where people are always downloading potentially
untrusted content from the network, and so having packages signed by a
single distro vendor where the public keys are appropriate configured
at install time makes a lot of sense.

On the other hand, if you are a commercial ISV providing a binary
distribution, every single distro has their own signing key, and they
aren't certainly going to be sharing it with the ISV, and if the user
is installing just one package from that ISV, the user probably
doesn't have the public key from that ISV yet, so getting a
certificate from a trusted source becomes an interesting problem.

That is not to say that the number of ISV's that might care about this
is non-zero, but to say that LSB's use of the RPMv3 package format is
fatally flawed and that there is zero benefit and we should just
remove it from the specification seems to be vastly overstating the
case.

> But I do ask the question:
>       What use is rpm as a LSB "standard" format if everything 
> in rpmbuild needs disabling?
>       Isn't cpio or tar or ... a better choice than a 
> lobotomized rpmbuild?
> 
> -------- JBJ quote ends

To answer the question:

1)  Package and filename registration
2)  Uninstallation
3)  Automatic running of post-install and pre-removal scripts.

These are all things that compressed tar balls do not provide.

(BTW, I checked --- post-install and pre-remove scripts *are* in the
LSB subset of the RPM format.  Trigger scripts aren't, but those are
less important for ISV packages, I would argue.)

> Ted's complaint about dependency specification inter ** 
> distribution ** is uninteresting, irrelevant and misguided; 
> the issue is what can an ISV reply upon when installing to an 
> LSB conformant distribution, exposing LSB managed names; and 
> the answer is LSB exposed 'Provides', administered in the 
> LANANA or in the spec document itself.

Sure, but in practice, the number of packages which an ISV can rely
upon is very small --- LSB, and packages provided by the ISV itself,
or other LSB-compliant packages, in practice.  There are very few of
those, and certainly having trigger scripts that do magic things when
*distro* provided packages are installed or uninstalled, whether it is
a new version of emacs (so that emacs lisp files can be compiled), or
when selinux is installed or uninstalled, (a) is very much distro
specific, and (b) in practice, most ISV's don't care about them.  This
is the sort of thing which is much more important to distro-provided
packages.

> Similarly, anonymous ISV's complaints of:  1: too hard, and 2: 
> not our build system and we support big iron Unix, are cited 
> by him and others as why 'yet another package builder' needs 
> to be written [int: it's already done -- see the JBJ post set 
> out infra], contra my express identification of two major ISV, 
> actively ** using ** RPM %pre and %posts [beyond the LSB 
> minimum required implementation].

RPMTAG_PREIN, RPMTAG_POSTIN, RPMTAG_PREUN, RPMTAG_POSTUN are in fact
required by the LSB.....  

> I _know_ how much history and hard 'dark corners' remain in 
> rpm package building.  Red Hat has gone through two succesors 
> to JBJ re-learning the lesson.  I found it again confirmed 
> doing task decomposition for Alien earlier this year, as I 
> blocked out the work there.  Dumbing down the RPM package 
> format to present a simplified form for the current Alien; 
> 'smartening' 'alien' to understand excised elements is to have 
> to recap a decade's work since RPM was done in perl. 
> Extending then to the later package format is more, but not 
> that much more.

No question --- packaging is hard.  As is source code management.  And
I note a very similar amount of... passionate.... debate on certain
mailing lists where Darcs users and former Arch maintainers will flame
on and on about how merging is hard, and tricky, with lots of "dark
corners", and how git and hg "just don't get it", because they don't
handle certain merges correctly.  They may be right, but git, hg, and
bzr are good enough, and so they are the tools that I use.  I
generally don't waste my time on those mailing lists either.

The question is *why* it we need all of those complexities to get all
of these corner cases correct.  If the needs for LSB's are fairly
simple, why use something more complicated, particularly if it has not
been adopted by all of the distributions ---- in particular the two
major enterprise distro's which most ISV's target their software at?

Your and JBJ's argument seems to be roughly, "If you aren't willing to
use Darcs, you should just go all the way back to RCS!"  Well,
sometimes git, or even svn, is sufficient for the job at hand.

I am quite willing to concede that the LSB-subset that we currently
have is the rough equivalent of svn, and maybe RPMv4 is more like
hg/git, and maybe rpm5 is more like darcs/arch/tla in terms of
sophistication.  But you know?  Lots of projects use svn, and they are
happy using svn.  I don't see the point of forcing them to use git or
hg, just as I generally ignore people who passionately argue about all
of the "dark corners" that darcs solves which git and hd do not,
because the way I use the tools, in practice they aren't an issue for
me.

And the good news is that everyone supports the RPMv3 format --- and
that is indeed a GOOD thing!  Creating a standard which would force
the enterprise distro's to upgrade to RPMv5 is just not interesting to
us.  Maybe that's what you and JBJ would like, since I'm sure you're
convinced of the innate superiority of the RPMv5 technology.  And
perhaps you're even right. But regardless, it's not clear these new
features that you and JBJ tout in RPM5 are useful for ISV's in the
general case.  (Or even the features implied by the RPMv4 package
format, for that matter.)

As far as your assertion that Edward Bailey's book documents the RPMv4
format --- it does not.  I just checked, and both the current and
"in-progress updated" version only describe the RPMv3 format.  The
more recent RPM Guide by Eric Foster-Johnson covers RPMv4, but it does
not describe the file format in any detail.  If you know of a
detailed, specification-level description of the RPMv4 file format, I
would be most grateful if you could point it out to me.  If it is not
a lot of effort to uplift to RPMv4 format, and it doesn't get away of
our other priorities, I'm not against it.  But from where I stand, if
we need to tease out the file format out of sources, and then when we
inevitably get it wrong, we get flamed to a crisp, we might be better
off fixing the known bugs that you have so kindly filed, and creating
a simple tool, and then just being done with it.


> JBJ reconfirmed to me today, post call that reaching rpm5 to 
> build '--lsb' packages is readily doable, and actually update 
> his earlier email for some changes in 'genpkglist' / 'rpmrepo' 
> space since the following piece:
> 
> 15:07  jbj> orc_orc: rpm2lsb is quite doable. meanwhile I
>  	haven't finished because there is no detectable
>  	interest. but I can/will knock out the conversion if
>  	desired.
> 
> Seems silly not to talk with him.  In part that's why I hang 
> on in the calls.  ;)

An rpm2lsb conversion tool is certainly interesting, but that's
actually not what we were talking about.  What we were talking about
is something relatively simple that takes a directory hierarchy, and
either (a) an simple XML file that contains the
post-install/pre-removal scripts, version information, and a few other
very basic information that would go in the tag format, OR (b)
command-line options that would provide this information.

So there would be no spec file, since at least some ISV's have found
it hard to deal with converting their build system to use it, and just
something that after their standard build process does a "make install
DESTDIR=/tmp/destdir" (or whatever), this tool would effectively
replace generation of the tarball with something which is effectively
a "smart tarball" (RPMv3 with post/pre scripts, etc.)

Sure, it could also be done by cons'ing up a fake spec file to fake
out rpmbuild, but given how simple the RPMv3 format is, this wouldn't
be hard to do directly.  Indeed, it probably could be done as a simple
perl script.  It indeed might be possible that ISV's won't find this
useful, either.  Clearly some ISV's won't bother, and will continue
using commercial installers from companies like InstallShield.  But
maybe creating a tool which makes it dead easy to create RPMv3
packages would be useful.  If it isn't, we may very well indeed drop
RPM from the LSB specification altogether.

Regards,

					- Ted