[cgl_discussion] Re: device enumeration

[Steven Dake]

Greg KH wrote:

>On Fri, 07 Feb 2003 12:13:56 +1100, Steven Dake wrote:
>  
>
>>I still haven't :) come to my senses that /sbin/hotplug is the right
>>solution.
>>    
>>
>
>I'm sorry, but I think you've lost this argument :)
>
>  
>
>>There are several problems with /sibn/hotplug which are 1) a
>>process must be spawned to handle an event
>>    
>>
>
>Linux spawns processes faster than any other OS.
>
>  
>
>>2) events take a long time to process through a shell script
>>    
>>
>
>So use a binary /sbin/hotplug if you care about speed.
>
>  
>
>>3) even if /sbin/hotplug is binary C code it would still have 
>>poor performance characteristics
>>    
>>
>
>Who cares about performance, this is _not_ a preformance criticial
>situtaion.
>  
>
Performance is critical; just ask any vendor.  It may not be critical in 
PCI hotswap and it certainly isn't critical in USB hotswap, but in 
next-gen architectures, the difference between speed would allow an 
operator to remove a physical device before the OS is ready to have it 
removed (surprise removal).  This works fine in pci (reads 0xff), but 
other architectures don't like it so much.  I am working on surprise 
removal support for advanced tca, but its much more complicated then an 
expected extraction and until I can say for sure it will work, 
performance does matter.

>  
>
>>4) calling a user
>>space application from the kernel to process any sort of event, besides
>>init, is just wrong, wrong I tell you.  I much prefer the idea of a
>>hotswap manager user space application executing select() on a character
>>device node and reading events via read or ioctl from the device node
>>when a hot insert or removal event occurs.
>>    
>>
>
>Sorry, but that's where you differ from the other kernel developers.
>/sbin/hotplug is conceptually much cleaner and nicer than having to 
>do select() or ioctls.  Remember ioctls are basically depricated, and
>you should not add new ones.
>  
>
I don't necessarily agree that _all_ kernel developers believe ioctl's 
should be deprecated.  Just look at all of the rich ioctls in the kernel 
currently.  The major problem without using ioctls (by using a 
filesystem for accessing methods in the kernel) is that there is no way 
to retrieve a return code.  Without a return code, how is the 
application supposed to know what the kernel did was successful, but by 
polling its state again?  Then the application may understand the 
operation was faulted, but the exact failure reason is still up in the 
air.  I suppose if the community makes the decision that living without 
return codes is acceptable, I could live with it.

>  
>
>>This is exactly what we do for IPMI hot insert/remove events for our
>>ATCA-like chassis, and it works quite well and fast (20 msec removal
>>times from event to device removed from OS) without process spawn
>>overhead.  This same solution using /sbin/hotplug would take 45 msec (I
>>tried it out), and require us to include more event processing code in
>>the kernel then we would need.
>>    
>>
>
>Who cares about the speed here?  I remember some discussion about speed
>issues with regards to hotplug unplug disks, but that turned out to not
>be a realistic issue (basically it was lazy testers) and not a real
>requirement.
>  
>
I was the person that raised this issue, and it isn't lazy testers.  It 
is the typical 10$/hour worker pulling these boards out in the telco 
environment.

Here is how it works.  A telco has a fault in their system.  They figure 
out what the exact failure is (bad switch, bad hub, bad disk, bad cpu 
blade, bad whatever), and they dispatch a 10$/hour worker to fix it. 
 The worker presses the hotswap request button, and while the OS is 
busily executing /sbin/hotplug, the worker thinks its ok to remove the 
device (when the OS still is using it).  This may "work" in Linux for 
PCI, but it certainly isn't correct that a device driver should expect 
0xff to be returned on pci operations (in the case of PCI).  Other 
architectures don't return anything indicating any failure causing real 
confusion.

Performance is _so_ critical here, because if the removal operation is 
fast enough, there is no phsyical way to remove the device from the 
slot/bus/whatever before the OS has removed the device from the 
operating system data structures.

>  
>
>>Mark Bellon (mbellon at mvista.com) has implemented a user space database,
>>API, and event transfer mechanism that sounds similiar to what your
>>planning and I'd suggest talking to him about what we are doing in this
>>space.  I really believe it would be in everyone's interest to work
>>together towards a common solution that fits USB/PCI as well as CPCI and
>>ATCA that keeps in mind the 4 points above about why /sbin/hotplug is
>>inappropriate for fast event management.
>>    
>>
>
>I've been in contact with Mark in the past, and it's my fault that I
>haven't followed up with him on some of the issues we've talked about.
>
>The biggest issue with mvista's implementation is that it's built on
>top of the chassis code, which is (imho) a horrible piece of code.  It
>also doesn't take advantage of the driver model, which is the way to
>go in 2.5 (yes, I know your implementation is only for 2.4, but sysfs
>can be backported if you want to...)
>  
>
Yes I agree sysfs/taking advantage of the driver model is a superior 
choice to mvista's chassis manager, but hey, we had to work with what we 
had available at the time.  If sysfs were in 2.4, we would have used 
that instead.  In future revs, we may backport sysfs to provide this 
sort of functionality and ensure that HDI works for both 2.4 and 2.5 
easily without a bunch of changes to parsing driver model information.

The key difference between MontaVista's HDI and whatever anyone else is 
working on is the excellent mechanism by which insert and remove events 
are transmitted (via the event broker) without the need to execute any 
type of hotplug scripts.  Perhaps both mechanisms could be used in 
MontaVista's implementation and let selection take its course.  This 
would allow us to keep the userspace database/api/tools/etc without the 
need to reimplement what everyone already agrees is the correct solution.

Thanks!
-steve

>So in short, /sbin/hotplug is here to stay and speed doesn't really
>matter :)
>
>thanks,
>
>greg k-h
>
>_______________________________________________
>cgl_discussion mailing list
>cgl_discussion at lists.osdl.org
>http://lists.osdl.org/mailman/listinfo/cgl_discussion
>
>
>
>  
>