[lsb-discuss] RPM decision process
R P Herrold
herrold at owlriver.com
Wed Apr 16 12:45:06 PDT 2008
> Jeff: won't solve this on the call; can Russ send a summary
> of issues?
It turns out that a 'review of the bidding' and open issues is
something I took the time to do in some detail. sadly, I fear
it does not bode well for LSB's relevance in 'packaging'
space.
http://mail.freestandards.org/mailman/listinfo/packaging is
dead to me [the connection times out], and so I cannot post
the 'permanent' URL for this summary piece from Mats a while
back; I also have some pieces in my private archive from some
participants I cannot republish as they did not cross a public
list. -- I set a rather nice full summary from Mats out below
at the bottom of this email.
As to some matters raised in the call, ...
My opening remark to the comment that 'We agreed at the face-
to-face that RPM package version format uplift was not
important and might be deferred' was -- Either uplift to RPM
package version 4, or just use tarballs [ar, cpio, XML blobs,
whatever]; RPM package version 3 is long obsolete,
incompletely protects the content, and is subject to a known
modification attack which cannot be detected in the way the v
3 signing leade is used.
Preapring this email, I had forgotten, but the essence of the
question I asked, and that Ted and I went back and forth on,
was repeated by JBJ working through support for a Debian user
of the LSB builder:
http://rpm5.org/community/rpm-lsb/0006.html
JBJ noted:
No matter what, all this stoopid configgery to produce a LSB
"standard" package could be simplified to a --lsb option in
rpm if anyone cared to ask me to implement.
No one has.
But I do ask the question:
What use is rpm as a LSB "standard" format if everything
in rpmbuild needs disabling?
Isn't cpio or tar or ... a better choice than a
lobotomized rpmbuild?
-------- JBJ quote ends
... my apology on the brain-dead color scheme at that site -- I
complained, but the maintainer likes it ;(
I had upstreamed the LSB RPMBUILD variant's errors at:
http://bugs.linuxbase.org/show_bug.cgi?id=1875
which remains open.
The RPM package version 4 has been documented for years -- at
least since 2004 in Max RPM on-line:
http://www.rpm.org/max-rpm/s1-rpm-file-format-rpm-file-format.html
in the revision to Ed Bailey's initial Maximum RPM, done, as I
recall, by Paul Nasrat when Paul was then employed at Red Hat
(and not the dreaded JBJ ;) ]
The ad hominum BS which I heard on this call: 'I saw Jeff
Johnson wrote it and so I stopped reading' is rejected as an
approach by me, and I trust by the LSB as a matter of policy.
On the call for paid staff of LF to ask me, a volunteer to
respond by writing an already existing set of documentation
and code, to what at the end of the day are simply uninformed
and inaccurate 'facts' about the RPM package format is
extremely off-putting to me.
There has plenty of distro-specific blame to go around in
packaging space discussions, but the LSB seems to fear
acknowledging its antecedents -- the circumlocutions another
call member used today to point fingers encourages 'in the
know' v. 'outside' cliquishness: I find in my archives from
the packaging list:
Date: Tue, 20 Jul 1999 16:26:59 +0200
From: Wichert Akkerman <wichert at soil.nl>
Reply-To: packaging at rufus.w3.org
To: packaging at rufus.w3.org
Subject: Re: [packaging] packageformat spec
Previously Jeff Johnson wrote:
...
> Can we agree on common semantics for file dependencies?
Probably, but Debian will never use them.
--------------------------
Amateurs painting bikesheds. ;(
Ted's complaint about dependency specification inter **
distribution ** is uninteresting, irrelevant and misguided;
the issue is what can an ISV reply upon when installing to an
LSB conformant distribution, exposing LSB managed names; and
the answer is LSB exposed 'Provides', administered in the
LANANA or in the spec document itself.
Stuart Anderson took a swing at that ** distribution **
level issue:
http://bugs.linuxbase.org/show_bug.cgi?id=967
Still open
LANANA is still dead:
http://bugs.linuxbase.org/show_bug.cgi?id=1279
Still open -- my ping after the 'yes it should be fixed' LSB
call in January, its last activity
https://lists.linux-foundation.org/pipermail/lsb-discuss/2008-January/004513.html
https://lists.linux-foundation.org/pipermail/lsb-discuss/2008-January/004515.html
----------------- quote
Russ: LANANA? Dan: we engaged them, got some responses; need
to update. What do people want? Proposed that the LF
administer, the original people set policy. Russ:
responsiveness is the major issue, bug open for a year and a
half. No visibility; no tracker, no way to know that they're
being listened to. Dan: possible political issues. Jeff: had
suggested a mailing list, seemed acceptable, never got done.
----------------- quote ends
Pretty much like the RPM package format uplift, I fear.
Similarly, anonymous ISV's complaints of: 1: too hard, and 2:
not our build system and we support big iron Unix, are cited
by him and others as why 'yet another package builder' needs
to be written [int: it's already done -- see the JBJ post set
out infra], contra my express identification of two major ISV,
actively ** using ** RPM %pre and %posts [beyond the LSB
minimum required implementation].
1. 'Too hard' is just sloth, pure and simple. It takes me
perhaps ten minutes to package a new random tarball, and less
to get a result back from the buildroot. For an ISV
application ** I ** ship ['... I'm not just a member of the
hair club ...' from the distro side], we have an automated
local builder which works with the rpm packaging system --
writing the Debian, OS/X and RPM based scripts for test
builds, took not more than a day.
ftp://ftp.owlriver.com/pub/local/COLUG/
shim-OSX.sh
shim-builder.sh
shim-debian.sh
And because an upstream certifying authority could not read
and understand error messages from the package management
tool, I tossed a scriptlet to inventory build environment
defects earlier today, and have upstreamed a report:
shim-inventory.sh
2. is easier -- OK -- don't be LSB certified in an RPM
distribution form; nothing mandates it -- use tarballs, or
carrier pigeons with clay tablets as they prefer.
I consciously asked posed question of 'futures' to Red Hat
yesterday, because two years later changed to rpmbuild are
still not defined into a Requirements document ** I ** could
implement from:
https://lists.dulug.duke.edu/pipermail/rpm-maint/2008-April/000883.html
I _know_ how much history and hard 'dark corners' remain in
rpm package building. Red Hat has gone through two succesors
to JBJ re-learning the lesson. I found it again confirmed
doing task decomposition for Alien earlier this year, as I
blocked out the work there. Dumbing down the RPM package
format to present a simplified form for the current Alien;
'smartening' 'alien' to understand excised elements is to have
to recap a decade's work since RPM was done in perl.
Extending then to the later package format is more, but not
that much more.
Two of the commercial Enterprise Linux distro vendors may
muddle along with locally maintained rpm forks which are
adding extensions only (not re-building that wheel), but
clearly also Wind River (a greenfield move), and EMEA Cable
and Wireless' OpenPKG (a fork and merge move) distribution
moved to the fully interoperant rpm5 codebase (Wind being
mentioned earlier in the call as one of the three CGL vendors
of consequence and to whom the LSB considers). CentOS has it
easy; we readily revel in slavishly replicating our upstream's
errors in the main line.
I don't mind a scuffle, but when I bring facts in, and get
argument based on opinion, raised voices and interruption, and
no details in response, I cannot give it credence.
Conspicuous by their absence in these calls and without any
interest in advancing away from the LSB 'RPM pkg version 3
with exclusions' [it appears, now a NINE and not eight year
wrong format], to RPM package version 4, are representatives
from the vendor which controls rpm.org -- no surprise; nothing
new. They are the market leader and can wait their opponent
in the commercially supported Enterprise Linux market out.
I reject the patronizing implied criticism of 'Johnny come
lately-ism' that I should have traveled to Texas out of my own
pocket to this most recent face to face meeting 'where
decisions were made' in what Joe Barr's article this week
seems to think was closed process, just as I would reject any
suggestion to that effect for my absence at Berlin; I've done
the face to face and post meeting memo long since with a well
respected LF stakeholder's long term representative. My
concern w pkg v 3 is well known, long standing, and of record.
I feed the bug tracker, participate in the mailing list
(several actually, for many many years), and call into the
call as possible.
While composing this, I happened to get a call from a huge
customer of a LSB certified Enterprise distributor's product,
and asked: 'How important is LSB certification on your
purchasing bullet list for products.' Sadly the answer for
them is: Commercial SLAs and support matter; the LSB
certification is not even on their radar.
I am not sorry for speaking frankly -- Please: do RPM right,
or stop doing it; to the extent that the conference call
bridge lags resulted in 'talking over one another' and made
for poor communication, I trust those on the call understand.
As promised, two email from my archive in this space follow,
as requested, with a bit of commentary in between from me.
-- Russ herrold
614 488 6954
---------- Forwarded message ----------
Date: Fri, 25 Feb 2005 06:58:26 -0800
From: "Wichmann, Mats D" <mats.d.wichmann at intel.com>
To: Stuart Anderson <anderson at netsweng.com>, R P Herrold <herrold at owlriver.com>
Cc: lsb-wg at freestandards.org, rpm-devel at lists.dulug.duke.edu,
packaging at freestandards.org
Subject: RE: [Lsb-wg] Re: [Packaging] Anybody is alive here ?
If the packaging group wants to restart some of its work, that's fine,
but a longer term question.
I do want to reemphasize what Stuart said, I'll try to be less clumsy
than I was in a comment in the bug-report.
At some point in the historical past, the LSB decided to settle on a
packaging compromise: specifying a format for packages, but not a tool.
The format IS the rpm format, but somewhat restricted. It's
documented elsewhere how restricted (the packaging spec is the
best place to look). LSB conforming systems are required to be
able to install packages in this format, which can certainly
be met by using rpm as the package manager, also converting to
other format with alien, and presumably there are additional
options we don't know about - one *should* be able to meet the
requirement on conforming implementation of being able to
install LSB packages by implementing an installer tool to the
requirements in the spec. [Note: set aside whether that's a
good idea in practice; that's what a complete spec is about,
you should be able to implement entirely to the spec and not
look at existing code to make things work].
Generally speaking, this is the LSB model: carve out some
common chunk of technology that everyone can agree on, even if
that needs to be a subset of a widely deployed piece of
software. For example, the LSB describes the set of services
provided by glibc that are described in the POSIX
specification, but generally stays away from additional
non-POSIX functionality that may also be present in glibc. As
long as developers avoid the non-specified functionality
(which we provide some tools to help with), they're able to
conform to the LSB spec.
The difficulty here is that now that a bunch of research was
done with our tool which checks packages for conformance to
the spec, it appears that packages constructed using current
distro versions of rpmbuild do not match the LSB spec, having
a fair bit of extra stuff in them. Sure, some of that is
sloppy packaging as was written somewhere in this thread, the
research didn't limit to carefully constructed packages that
try to be LSB conforming.
So the question is, can we make the subsetting model work in this
case? As Stuart wrote,
"To solve this, there are 2 paths we can take
1) Fix the spec by adding in everything that is missing
2) Fix the tools so they only output what is required"
That is, increase the size of the subset, apply some sort of
restriction in LSB mode as we've been able to do elsewhere, or
maybe some combination of the two. Stuart has talked already
about problems with #1 in that the missing stuff seems to fall
outside the "common stuff that everyone can agree on", and the
problem with #2 is we're not quite sure how to do that. Which
left us /considering/ whether it made more sense to develop a
new tool which could meet path #2.
Maybe some wiser folk than us on the rpm list have
suggestions? Could rpmbuild be taught to limit what it
outputs? One concern I have with modifying rpmbuild is getting
it in the hands of LSB developers: current deployments of rpm
range from 4.0.x on up through 4.4 which I gather is the
current codebase head, and there are an awful lot of systems
pegged onto the 4.1-or-earlier track. But maybe that's a
later question. Could it even reasonably be done?
================================= Mats' piece ends
The transforms for 'bowdlerizing' ** cough ** building
packages are well known into the LSB format, and as it turns
out, JBJ has them in the RPM5 fork already. This was posted
to the 'vendor neutral' RPM maintenance list set up by Red Hat
at Duke by their proxy. There is no need to re-do what has
already been done, unless of course JBJ's LGPL work is for
some reason not functional. Redoing it yet again and
differently at LF is a re-invention of a tool, and fool's
work, as I remarked in the call today.
JBJ reconfirmed to me today, post call that reaching rpm5 to
build '--lsb' packages is readily doable, and actually update
his earlier email for some changes in 'genpkglist' / 'rpmrepo'
space since the following piece:
15:07 jbj> orc_orc: rpm2lsb is quite doable. meanwhile I
haven't finished because there is no detectable
interest. but I can/will knock out the conversion if
desired.
Seems silly not to talk with him. In part that's why I hang
on in the calls. ;)
Date: Sun, 3 Sep 2006 17:27:55 -0400
From: Jeff Johnson <n3npq.jbj at gmail.com>
Reply-To: RPM internals development and distro coordination
<rpm-devel at lists.dulug.duke.edu>
To: RPM internals development and distro coordination
<rpm-devel at lists.dulug.duke.edu>
Subject: rpm-devel] LSB packaging anyone?
On Aug 23, 2006, at 10:42 PM, Jeff Johnson wrote:
> Hmmm, now where was I? Ah yes ...
>
> At OLS this year I finally met Mats Wichmann, where the
> subject of LSB packaging was discussed a bit.
>
> What's sad, both then and now, is that LSB "standard"
> packaging is simply not yet possible to produce easily and
> reliably
>
> While there are many flaws in the LSB packaging standard
> (imho), it would be rather easy to make rpm packages that
> are conformant with the existing standard(s).
>
> Basically, all that is necessary to convert most any
> existing rpm package to LSB standard packaging is:
> a) discard all dependencies except "LSB" and "lsb-*" from headers
> b) discard any/all trigger scripts.
> c) rewrite headers with absolute file paths instead of the
> {dirname,basename,dirindex} that has been used in rpm many
> years now.
>
> I can write that conversion program in a day or so whenever.
>
I managed to scrounge a couple of hours to get the
preliminaries for a rpm2lsb conversion utility together. ATM,
the utility takes 0 or 1 argument, and spews the signature and
metadata headers in YAML. Yawn, work-in-progress in the tools
sub-directory from rpm CVS on the rpm-4_4 branch.
The rpm2lsb utility (and its inverse lsb2rpm) are the first in
a series of package conversion utilities that are going to be
added to RPM when finished.
AFAIAC -- as the pretender to RPM maintainer -- I hereby
officially bequeath the format documented in "Maximum RPM" to
LSB for whatever purposes they see fit.
So, the format formerly known as "rpm-3.0.x" is officially
renamed as "LSB format".
The format will change only by act of LSB, with versioning in
rpmlead and dependencies tracked by LSB, not rpm.
Ptooey! R.I.P. Have fun!
Seriously, the "LSB format" is deeply flawed mainly because
the development paradigm and needs in 1997 was considerably
different than today's.
I can still hear ewt say
If there's something wrong, segfault immediately.
Never check an error code you can't handle. but that's just
rpm's %ghosty and hysterical past.
> What I've been muddling since OLS is whether to add a --lsb
> option to rpmbuild to accomplish LSB packaging instead. I
> currently lean towards a conversion utility rather than a
> build option (which is even less than a couple of hours of
> work), because LSB packages lack sufficient information to
> be installed on most current linux distros, and I'd rather
> not see rpmbuild --lsb be used to, say, produce packages
> without dependencies that cannot be installed reliably.
>
Not forgotten, just not yet. A conversion utility outside of
rpmbuild first.
> Is there any interest in producing LSB "standard" packaging?
>
I am, and LSB seems to have some interest as well. Anybody else?
73 de Jeff
=================================================
More information about the lsb-discuss
mailing list