[cgl_discussion] device enumeration

Steven Dake sdake at mvista.com
Thu Feb 6 17:13:56 PST 2003


Patrick Mochel wrote:

>On Thu, 6 Feb 2003, Steven Dake wrote:
>
>  
>
>>In the current implementation for ga mapping (mapping a scsi device for 
>>a disk in a certain slot/chassis), we do use devfs.  I've come to my 
>>senses and I am going to change this in our CGE 3.1 product to make 
>>enumeration of those devices take place in userspace without the need 
>>for devfs.  Also, the work I have done on our automatic storage 
>>multipathing enumeration occurs in userspace, following this philosophy. 
>> The work Mark Bellon is doing for device enumeration (the next-gen 
>>MontaVista HDI work) is based in userspace and does not depend on devfs 
>>at all.
>>
>>devfs is an interesting idea, but controlling naming/permission policy 
>>in kernel has its share of problems.  The only downside to controlling 
>>permission/naming in userspace is it is currently done in a static form 
>>via MAKEDEV which is inapproriate for a variety of reasons.  
>>    
>>
>
>I am glad to hear that you're moving away from devfs as a solution. :) 
>
>Maintaining that policy in the kernel is definitely the wrong thing to do. 
>By keeping that policy in userspace, you are able to extend and scale 
>almost without bound. So, we agree. :) 
>
>However, I don't understand what you mean by 'static'. Solutions that were 
>discussed at the kernel summit, and in private conversations with Greg KH 
>and Linus, involved having some sort of userspace database, which could be 
>updated by a user at any time. 
>  
>
static as in MAKEDEV makes static nodes such as /dev/sda which can 
change between reboots/hotswap events and are not really usable in any 
sort of hotswap environment.  I think the database is the way to go and 
that is what we are doing at MontaVista (see below).

>Although, the permissions would only be set once, when a device node was 
>created (e.g. after a device was inserted). Subsequent permission changes 
>would not automatically modify existing permissions on device nodes 
>(though this a behavioral issue of the agent providing the permission-
>modifying functionality). 
>
>  
>
>>This is why a next-generation HDI solution is needed to solve the naming 
>>problem in userspace, not in the kernel (with some kernel help to tell 
>>userspace of insertions/extractions that the hardware may be aware of).
>>    
>>
>
>This help, I presume, is /sbin/hotplug. Greg and I have been working on
>infrastructure to provide a good, mainstream solution for persistant
>device naming using sysfs and /sbin/hotplug for 2.5/2.6. I hope that we
>can align our efforts to provide a single solution for this problem..
>
>	-pat
>
>
>  
>
I still haven't :) come to my senses that /sbin/hotplug is the right 
solution.  There are several problems with /sibn/hotplug which are 1) a 
process must be spawned to handle an event 2) events take a long time to 
process through a shell script 3) even if /sbin/hotplug is binary C code 
it would still have poor performance characteristics 4) calling a user 
space application from the kernel to process any sort of event, besides 
init, is just wrong, wrong I tell you.  I much prefer the idea of a 
hotswap manager user space application executing select() on a character 
device node and reading events via read or ioctl from the device node 
when a hot insert or removal event occurs.

This is exactly what we do for IPMI hot insert/remove events for our 
ATCA-like chassis, and it works quite well and fast (20 msec removal 
times from event to device removed from OS) without process spawn 
overhead.  This same solution using /sbin/hotplug would take 45 msec (I 
tried it out), and require us to include more event processing code in 
the kernel then we would need.

Mark Bellon (mbellon at mvista.com) has implemented a user space database, 
API, and event transfer mechanism that sounds similiar to what your 
planning and I'd suggest talking to him about what we are doing in this 
space.  I really believe it would be in everyone's interest to work 
together towards a common solution that fits USB/PCI as well as CPCI and 
ATCA that keeps in mind the 4 points above about why /sbin/hotplug is 
inappropriate for fast event management.




More information about the cgl_discussion mailing list