[Openais] Proposed change to default nodeid generation

[Andrew Beekhof]

On Aug 20, 2008, at 11:41 PM, Joel Becker wrote:

> On Wed, Aug 20, 2008 at 11:17:18AM -0700, Steven Dake wrote:
>> On Wed, 2008-08-20 at 18:52 +0200, Andrew Beekhof wrote:
>> We talked on irc and came to the conclusion that apparently it is
>> impossible to fix the kernel issue with the assumption of 1^31 bits  
>> and
>> it is impossible to fix the openais/corosync assumption of 1^32 bits
>> nodeids.
>
> 	Boy, are we awesome.
>
>> Therefore a patch (with man page) of an api that gets the nodeids  
>> in 31
>> bit fashion from cpg would be workable.
>>
>> ie:
>>
>> cpg_nodid_31bit_set(handle).
>>
>> would be suitable if this would meet everyone's needs.
>
> 	Well, what happens with CLM?  Does it report the 32bit nodeid?
> What about EVS?  Btw, it's interesting (though irrelevant) that EVS  
> uses
> 'unsigned int' instead of uint32_t.  I think this is a little more  
> core,
> and I'd like to explore what we're really after.
> 	I don't think we should force all openais/corosync nodeids to
> 2^31 bits.  The protocol should stay 2^32 bits.  I don't think Andrew
> was asking for this either.

Correct - this is why no header files were part of the diff.

The protocol should clearly not be touched and I was only proposing
that the autogenerated nodeids be limited to the range 0..2^31 instead  
of 0..2^32.

This is all the patch was attempting to change, I apologize if the  
intent
did not match the implementation.

> 	I think Andrew was just looking for a way to limit the
> autogenerated node numbers,

Right.  Although we may have to tell users not to use it  
(autogeneration)
since the the values it generates aren't supported by the rest of the
RedHat's stack - particularly the kernel portion.

> because his software uses the autogeneration.

Not so much "uses" as "lets openais do its thing because its none of  
our business" :-)

> The cman stack never cares because it sets the nodeids
> itself, which requires a configuration file and explicit nodeid
> specification by the admin.
> 	So, first, Andrew, does pacemaker have a method for querying
> membership and nodeids?  Forget ocfs2_controld/dlm_controld.  I just
> write a random application, and I want it to use pacemaker.  Are there
> functions for "what is my nodeid?" and "what nodeids exist in the
> cluster?"?  What do those functions look like?  I'm assuming they work
> in the absence of cpg, so it isn't just cpg here.

Right.  Pacemaker has such functions but they all use uint32_t.

It's not my code that requires this change.

>
> 	Both Andrew and Steve, I'm assuming that pacemaker exists as a
> kind of corosync plugin these days.  So it has some access to the  
> config
> space.  Now, the cman config plugin fills in the nodeids, whereas
> pacemaker appears to get them from corosync's automatic method.  Is
> there any way a plugin could have a hook from the autogeneration code?
> That is, corosync determines it should be autogenerating a node  
> number,
> so it calls ->auto_node_num().  This defaults to the current IP  
> address
> method, but pacemaker could provide a new method that returns whatever
> value it wants.  Internal to corosync and on the wire, it's still
> stored/sent as uint32_t, just like cman's nodeids.

That feels like a layering violation to me.

It's not a cluster manager's business to dictate nodeids to the
infrastructure layer, it should be telling us (as it does now).

It also presents problems when/if different pieces want to use
different allocation schemes.