[cgl_discussion] CGL

[Brian F. G. Bidulock]

Mika,

Why do you refer to kernel developers as "you people"?  And yet, in
another breath refer to yourself as a "Linux advocate".

Just an observation, but it appears to me that the tone of your
remarks are at cross-purposes with your stated objective.

Referring to any group of people as "you people" is antagonistic and
certainly not conducive to an atmosphere of cooperation.

As a "Linux advocate" perhaps you should ask yourself why advocates
do this or say that, instead of asking why "you people" do this or
say that.

Please see some (more heated) comments below...

--brian


On Mon, 26 Jan 2004, Mika Kukkonen wrote:

> Well, you have invoked the name of "CGL" on this mailing list, so
> speak of the devil ... 
> 
> Seriously, one thing that has annoyed me beyond reason is that people
> keep telling me that "CGL is evil because I read it somewhere", but when
> I try to find where they read it, I almost always fail.
> 
> So may I suggest, that if you have any issues about CGL you want to 
> discuss, you would CC cgl_discussion at osdl.org and/or me, or is that
> too much courtesy to ask?
> 
> I am going to reference Jes's email, as it seems to be most on topic
> of all items so far on this list:
> 
> > Let me try to make a long story short. Keep in mind that this is how I
> > saw the whole CGL process perform, people who were closer to it might
> > have a different view.
> 
> yes, I do :-)
> 
> > Basically how it went on was a closed forum with people from various
> > industry groups sitting down behind closed doors writing a large spec
> 
> Let me give you some history: CGL and it predecessors existed way before
> it was announced to be an OSDL working group January 2002, and before
> that it _was_ a very closed and selected group of companies having the
> same itch (namely use of Linux and OSS in telecom industry) and 
> wondering how to scratch it. So it is no wonder, that it has taken some
> time to transform the mind set of some management people to a more open
> direction.

"More open"?  I always though that software was either open or closed.
I didn't think that there were degrees of openness.  Maybe there are,
but I think that purists would agree that software is either open or
not.

Or were you referring to mindset as in open-minded?  The same is
probably true: open or shut.

Either way, what makes you think that several corporate management
team's personal trials balancing greed and guilt would mean anything to
the open source community?  Did you have a point here?

> 
> > of 'we want this this this and this' and then publishing it with a
> > 'please implement all this for us'. Because it was done in the closed
> > there was little to no communication with the kernel community, in
> 
> The CGL V1.0 doc reads like a shopping list, _because_ it is a shopping
> list! It really was made with the mind set that it is a list of
> requirements the telco companies would like to see the distro companies
> to implement. 

Pardon?  Which "carriers" are represented?  Please list them.  I see a
lot of large computer manufacturers that have been trying to pass off
"commercial grade" equipment on telecommunications companies for several
decades without much success.  I performed selection, acceptance and
installation on such equipment for quite a few years in the telco and,
believe me, I think we invented the term "Carrier Grade" to describe a
set of requirements that general purpose computer equipment manufactures
could never meet (and, in my experience, still don't).

> 
> You know, "implement these in your distro and we'll buy it" type
> of thing. There _never_ was an expectation that the people in CGL were
> hoping to get these features for free, and in fact around that time we
> felt we would do many of those features ourselves (aka TLT/CGLE ...).

I believe we have had this discussion before, but allow me to try again:
if you want to know what a "carrier" will buy and what constitutes
"carrier grade", ask the carrier.  If the carrier is interested in
purchasing from you, you will receive, oh, about a 400 page RFP laced
with Telcordia, ETSI, ITU and other International and National
requirements.

For North American operation, computer systems must normally at least
conform to the requirements in the OSSGR.  Need I mention things like
M.1030.

It strikes me that, maybe due to the initial participation in the
specifications, that the spec represents more what a commercial computer
vendor would like to pass off on a carrier, rather than what carriers
require from all central office systems.

I don't mean to rant, but I know of at least one commercial computer
vendor that doesn't seem to understand that the lights go on the front
and the wires on the back.

> 
> What comes to communication, it is a two way street, and we are all 
> aware how ... interesting discussions on controversial topics can get
> on LKML, and really, if you look at the _kernel_ features in V1.0, there
> really is not much to write to LKML about in first place.
> 
> > particular a lack of discussions about what is feasible and realistic
> 
> Please feel free to give me some examples of priority 1 features that 
> you feel are unfeasible and unrealistic.

Almost all carrier requirements are systemic.  That is, they are
requirements placed upon an assembled system with a specific software
load.  Carriers do not build such systems from components.  They do
install, operate and maintain them.  Their cash flow runs through them.

Perhaps the single most stringent set of requirements for carrier grade
equipment is colloquially referred to as the "shake and bake", where a
system is shaken to simulate earthquake zones and melted to determine
maximum operating temperatures and burnt to determine the amount of
toxic fumes released by the components as they burn.  Of course, the
software must continue to operate to the culmination of this operation.
Commercial computer vendors were usually surprised that they were
expected to destroy several fully equipped systems (often valued at
millions of dollars) to certify their product.

To address this carrier grade requirement from a software only
perspective would be unfeasible and unrealistic.

Perhaps the single most stringent set of requirements address system
availability and reliability.  Carrier grade systems must be engineered
with no single point of failure (NSPOF) and all subsystems, whether
hardware, firmware or software are expected to be engineered with at
least 1:1 redundancy.  Many have heard of 5 9's (or 10 min a year) but
this only applies to core switching elements.  Core signalling network
elements (such as STPs) are subjected to availability requirements an
order of magnitude more stringent.  Many do not understand that these
availability requirements include software, an more importantly software
errors.  Let me focus on this last point.

Reliability calculations performed according to carrier requirements must
include the number of software errors that the system is going to
encounter during operation and the the effects of those errors (service
affecting, system affecting).  Can you tell me how many software errors
that a distro conforming to CGL specifications is going to encounter
during a given hour of operation?  The probability?  How drastic the
effects of those software errors are going to be?

This number 1 carrier grade requirement is not addressed by your
specification, and it is not feasible or realistic to do so for a
general purpose Linux distribution.  A company might freeze a copy of
the Linux kernel, GNU and other open software, perform rigorous testing
and derive a number for a specific load with a specific hardware
platform and a specific application, but your spec won't help them much.

One more example: BOC requirements (deriving from Telcordia OSSGR)
require that all system parameters have a default-default.  If you don't
know what a default-default is, you should probably be stripped of the
right to use the term Carrier Grade.  All system defaults in a BOC
software system (including kernels) must have a default value settable
by the operating company (carrier) that is used by the system on boot.
Not some: all.  That means that just about every meaningful #define in
the kernel would have to be turned into a variable, and that variable's
default value would have to read a kernel boot from a user-specified
list.  This certainly applies to just about every value in limits.h.
Where was that in your specification?

Is it feasible to convert all constants in the Linux kernel to variables
defined at boot?  Is it realistic to expect that to happen?

On the other hand, I see things in the spec that carriers care less
about: check-pointing is one, clustering another, high availability
another.  Carriers could care less how you meet your reliability
numbers, as long as you can meet them and the system has no single point
of failure.  This may be a (commercial computer) vendor-centric view of
what they have convinced themselves carriers want, but is out of touch
with the reality of what carriers ask for.   Have you even looked at the
OSSGR?  Don't take my word for it: take a look at the carriers own
requirements documents, standards and practices.  Perhaps you could have
saved yourself the (specification) effort.

> 
> > and in the little interest in adapting the spec once it was finally
> > published (presumably due to the we already spent a lot of time
> > wheeling dealing internaly over it, we don't want to change now).
> > This
> 
> Now you totally lost me there; "adapting the spec" to what? What I do
> agree with is that every committee type of work group (like CGL) has
> more latencies than a group lead my a single dictator (you know, review
> and voting stuff), but from the beginning we have been pining for some
> real feedback from anybody out there, and as far as I recall there has
> not been much technical feedback, just "we hate you, go away". The one
> exception I remember is HCH's feedback on LKML, and that was 
> professional and good.

If this is lost on you then perhaps you are indeed different than the
"you people" from which you exclude yourself.

Once of the great values that Linux and its developers provide is a
fresh look at old problems.  Sometimes the problem has been discovered
to be the requirement and not the solution.  Sometimes the interface
limits the implementation.  I have seen a number of occasions where
developers of the Linux kernel have adjusted an interface to adapt to a
solution implementation that better meets the original purpose.  By
laying a set of requirements, you blind the effort to this
purpose-driven solution process that seems so natural to kernel
developers.  (If you're an old fart like me, go back and read
"Breakthrough Thinking".  Younger folk learned that stuff in grade
school.)  Setting a requirements document focuses on the problem rather
than a purpose driven solution.  This happens because the requirement
masks the purpose.  With CGL this is even worse, because it appears
written to satisfy the vendors of a product rather than the users of it.

> 
> > resulted in some pretty serious flaming from the community once the
> > spec was finally out to the point where CGL is almost a swear word
> > amongst developers.
> 
> And this still puzzles me; why you people feel so threatened and 
> insulted by CGL? I mean all we have produced so far is couple documents
> that you, the kernel developers, can just ignore. So why all this hate?

When you refer to a group of people as "you people" is does not threaten
them, but it does insult them.

> 
> > Even for the people who wanted to try and contribute constructively,
> > it was extremely painful, one had to deal with silly silly paper forms
> > and snail mail just to get access to an 'Open Source' project.
> 
> The CGL Participation Agreement was created because OSDL felt that there
> needs to be an agreement that makes sure that all documents CGL creates
> are "free" of IPR problems, and it was also OSDL's idea to require this
> on paper. I'll let Tim to go into more detail on that, if he wants.

The flip side of that is that no IP when into it.

To prove that IP was contributed under the agreement would require
proving that the contributed actually owned the IP to which you make
claim.  A tough thing to show.  Also, transfer of IPR requires an
equitable contract.  What did OSDL pay for those rights?

As an individual, it wasn't the participation agreement (obviously
untested in court and pretty much unsupportable) that turned me away.  It
was the participation fee.  I figured that if I was going to contribute
to the effort and money was going to move at all, that it was flowing in
the wrong direction.  Sure I pay a participation fee to say, IEEE, and
surprisingly they don't require me to sign an IPR agreement, and pay a
registration fee for IETF Plenary sessions, but not a participation fee
to working groups, and they don't require an IPR agreement, but
encourage IPR statements.  Please don't try to allude that your work is
so gosh darn important that it would shadow the work of the IEEE or the
IETF and thus require such strong language or increased commitment.

Gee, I can jump in a start developing on the Linux kernel and might even
accomplish something even if I do stick my foot in my mouth twenty or so
times on lkml, for (almost) free!

> 
> > Ben really hit it smack on with his comment: "The whole 'closed
> > working group from big companies' thing makes me sick".
> 
> Again, why? AFAIK, some of you people write NDA's to be able to work
> on drivers, and some people work for distribution companies which also
> have to deal with these big companies, which btw are called "paying 
> customers". 

I've heard of NDAs for drivers, particularly for proprietary hardware
where the competition is stiff and the margins narrow, where giving away
too much information on a custom VLSI chip for the sake of Linux could
bet the farm in the remainder of the market.

I don't see kernel developers signing NDAs for user interfaces, or
worse: requirements documents.

> 
> I think it is OK to make fun of big companies and the stupid things
> they do because of stupid middle managers who do not understand 
> anything, but a sad fact of life is that a _lot_ of people get their
> salary either directly or indirectly from these companies, and my
> personal goal (as I still feel I am Linux advocate) is to direct some
> of that money these companies to the "Linux ecosystem", so that this
> wonderful thing called Linux and Open Source can continue to grow and 
> flourish. If that makes me an evil guy in your eyes ... oh well.

Please list the open source projects to which ODSL has directed money.
I would like to see that list.  (No, no, not restricted resources, but
actual money.)  'Cause, gee, I get the impression that you guys just
burnt 20-30 Million dollars at a time when the industry could least
afford it.  (That would buy away a lot of "I'll code for food." signs
from Preston and the LBJ.)

> 
> > So sorry if this sounds a little harsh, however I believe that this is
> > how a lot of people in the community feels about doing development
> > this way. It just doesn't work. Even if you have a brilliant spec once
> > you're done, you a likely to face a lot of resistance when you publish
> > it as the community in general dislikes to be told what to do.
> 
> On the other hand, does it feel nice to hear that some people like your 
> project and want to use it, and maybe in progress of doing it give you
> some feedback and even contribute patches back. What you people do not
> see is that there is _a_ lot of more in CGL than just me and bunch of
> people that participate in CGL meetings. A lot of people in the member
> companies are working on Linux and OSS related projects because of CGL
> existence. Of course it can always be argued that projects like OpenHPI
> (http://openhpi.sourceforge.net/) would exist even without CGL, but my
> point is that most of the CGL member companies are working on Linux,
> they just do not do it under CGL banner because of it's negative brand.
> 
> So one more time, it is very easy to say something is wrong, but it is
> much more difficult to say how it should be fixed and then do it. I am
> all ears for your ideas on CGL.

I'll give you some suggestions:

  1) Go find out what Carrier Grade means, because I'm am convinced that
     you do not know what it means.  Ask a Carrier.  Read the LSSGR and
     OSSGR.

  2) Scrap the POC concept, because compliance is not judged by carriers
     at the subsystem level, but at the system level.

  3) Scrap every requirement in the document that does not directly meet
     one of the carrier requirements.

  4) Discard the title page (because there will be no requirements left
     behind it that are not elsewhere documented, also, as piecemeal
     compliance is useless to a carrier, scrap the POCs which are
     basically the CGL version of the LSM).

I dislike the ODSL/CGL primarily because of the following:

  -  it does not meet nor assist equipment providers in meeting carrier
     (grade) requirements

  -  it hoards and restricts access to precious resources

  -  it is diffusive to development efforts that it pledged to support

But most of all, I dislike it because, at it inception, it appeared to
be a promise to fulfill the dream of every telecommunications engineer
(including me), but has wound up to be an embarrassment and a parasite.

Please ask your computer-vendor bosses to suck OSDL and CGL back inside
themselves to save us their shame.

I could go on, but I'm sure that I bored everybody in the first couple
of sentences, so I'll shut up now...  Sorry to preach.  I hope some of
it was to the choir.

--brian


> 
> --MiKu
> 
> 
> _______________________________________________
> cgl_discussion mailing list
> cgl_discussion at lists.osdl.org
> http://lists.osdl.org/mailman/listinfo/cgl_discussion

-- 
Brian F. G. Bidulock    ¦ The reasonable man adapts himself to the ¦
bidulock at openss7.org    ¦ world; the unreasonable one persists in  ¦
http://www.openss7.org/ ¦ trying  to adapt the  world  to himself. ¦
                        ¦ Therefore  all  progress  depends on the ¦
                        ¦ unreasonable man. -- George Bernard Shaw ¦