[Ksummit-discuss] [TECH TOPIC] Sensors and similar - subsystem interactions, divisions, bindings etc.

Laurent Pinchart laurent.pinchart at ideasonboard.com
Thu Jul 28 16:39:33 UTC 2016


Hi Jonathan,

On Wednesday 20 Jul 2016 22:18:11 Jonathan Cameron wrote:
> Hi All,
> 
> This topic would be around the way the various subsystems interact, in the
> rough area of 'sensors' (I haven't yet had much of an issue with subsystem
> crossing with output devices but maybe that's just over the next hill!)

It's already here :-) We have an API to control camera flash (as in light, not 
memory) in V4L2 and another one in the LED subsystem.

> Scope may well be wider but includes:
> * input (some of it)
> * hwmon
> * iio
> * comedi(?)
> * thermal
> * power/battery
> * gpio - the blur between gpios and beaglescope / PLC type I/O.
> * v4l - when does a device jump from being a multi pixel thermopile
> to a thermal camera? Smart fingerprint scanners etc.  Gesture recognition
> sensors (effectively 9ish pixel cameras)
> * Lots of random things we haven't seen yet.
> 
> All these make use of devices that are at their hearts ADCs. Whilst it
> would nice if the underlying devices always fell into one clear
> category that is not always the case. Hardware descriptions (e.g.
> device tree) are an important area of contention.
> 
> The examples that follow are mostly based around IIO interactions with
> hwmon / input as that's my area of expertise. However, I know that
> similar issues occur at other boundaries.
> 
> Guenter Roeck has been working on hwmon / thermal interfacing recently.
> We also regularly push drivers one way or the other around that boundary.
> 
> Dmitry and I often have to decide on the IIO / input scope boundary
> (and we've moved it over time) + a number of cross over drivers have
> turned up recently.
> 
> The other subsystems I encounter have been 'quieter' in their
> interactions with me but many of the issues below apply to them as well.
> 
> A rough list of related topics that might be worth face to face
> discussion includes:
> 
> 1) Are we happy with the somewhat adhoc divisions of subsystems as they
> stand?
> 
> I am personally happy enough with this, but it's worth checking
> everyone else is. This probably causes more pain for new submitters
> than it does for old hands. Perhaps we can come up with a checklist style
> doc to save people having to ask.
> 
> We have had quite a few cases where a whole driver is submitted to the
> 'wrong' subsystem. Not the best first interaction with the kernel
> world for new contributors. (This sort of area is attracts newbies
> as it's relatively simple and the hardware is fairly cheap and more
> than once their first experience with review as exactly this!)
> 
> If we aren't happy, what do we do about it?
> 
> I'm hoping we are happy or, at least, resigned to the current state
> of affairs, but think the question should be asked from time to time
> as the answer may well change + the discussion will highlight pinch
> points we can work on.
> 
> 2) Bridging drivers - there will always be cross over cases.
> E.g. Generic ADCs used for battery voltage monitoring.  We have a
> number of bridges in mainline already and others out of tree.  How
> important are these and what features would make them more useful?
> 
> Is it ever sensible to have two drivers for the same part because
> the use cases are so different? (I hope there is always a better way
> but maybe not)

I'd love to say no, but we have already hit that problem. There are two 
drivers for the ADV7511 HDMI encoder, one in the media subsystem for V4L2 
output devices (drivers/media/i2c/adv7511.c), and one in the DRM/KMS subsystem 
for display devices (drivers/gpu/drm/i2c/adv7511.c, being moved to 
drivers/gpu/drm/bridge/). Try compiling them both in the kernel and see how 
they race to probe the same device (enjoy the show, popcorns not included).

> This covers both generic bridges (e.g. iio-hwmon) and device
> specific bridges (a good example of a touch screen driver turned
> up in my inbox yesterday).
> 
> 3) Bindings (device tree and similar).  When it is appropriate to use a
> device tree to describe the overlap (bridging of channels)?
> Sometimes there is obvious real hardware involved (a thermistor on a
> generic ADC input for example), but there are many grey areas.
> 
> Finally bashing out an answer to the issue device tree maintainers
> have with the iio-hwmon bindings would be great.
> On a less biased note, that example is pretty fiddly to define but would
> act as a good basis to work from more generally.
> 
> We know we got it wrong (back in the dark ages), but it isn't obvious how
> to get it right!
> 
> IIO-input has been partly held outside mainline for years by this issue.
> 
> Also some practical design decisions to make around how to implement
> these mappings to work best with deferred probing etc.
> 
> 4) Should we drop all the bindings for bridging between such subsystems
> and do it all from userspace?
> 
> I think we may end up with a hybrid of the two, but need to be able to
> make it work in 'standard' cases without userspace being involved. That
> hybrid solution may well be devicetree overlay based... Unclear so far.
> + plenty of crazy things that 'might' make sense where there isn't even
> the pretence of representing real hardware.
> 
> 5) Complex device interaction usecases.  At the moment the ones I've come
> across are mostly contained in IIO.  The moment we start sticking in
> MUXes, AFEs (Analog Front Ends) and straightforward analog sensors in the
> mix it can get fiddly.  Swapping war stories may well be worthwhile on
> this. This stuff also turns up in ASoC for example so probably lessons
> to be learned from there.
> 
> The analog devices software defined radios are another possible case
> study.
> 
> There of course may well be lessons to be learned from similar
> interactions elsewhere in the kernel.
> 
> There is a lot of history in how we ended up where we are (it all made
> absolute sense at the time). Sitting back and taking the time
> to discuss the future would be great.  Whilst this might be solvable
> by email we've made no definitive progress for years
> (and what has been made has been on a case by case basis deep in
> driver reviews.)
> 
> I threw comedi in the list above but, at the moment, I think the more
> likely direction there is a single userspace library abstracting
> the underlying subsystem (Analog Devices are working in that
> direction - perhaps Lars can offer more on that?).
> 
> GPIO is another interesting case - a lot of hardware is capable of
> parallel sampling, some at high speeds. It's another area that
> is probably too specialist for this discussion, but if people want
> to dive into the details it might be interesting.
> 
> I think we have only a small amount of fuzz around the v4l boundary,
> but wanted to leave the door open if anyone wants to discuss that
> one further as it's come up a few times over recent years.

Don't forget to take system integration into account. If I given you a high-
speed ADC you will not think about V4L2 as your subsystem of choice. If the 
system designer has connected that ADC to a CPLD that generates fake 
horizontal and vertical sync signals, and connected the output to the camera 
interface of the SoC, you will be left with no choice but use the V4L2 API. 
That's largely a userspace issue in this case, but it implies that V4L2 need 
to define an "image format" for the ADC data.

> The SoC world is a major case of one device, many uses.  Some SoCs
> are turning up with multiple ADC units, sometimes with different
> designs, sometimes simply so that the same hardware can be used for
> different things.
> 
> A few suggestions for people:
> 
> Guenter Roeck (hmwon)
> Dmitry Torokhov (input)
> Me (Jonathan Cameron - IIO)
> 
> Someone thermal related (not sure who would be most interested)
> Someone power supply related? (Guenter, any suggestions?)
> 
> Someone device tree related (Mark Rutland? Rob Herring?)
> 
> Linus Waleij (gpio)
> 
> Mark Brown - as some of concepts of IIO bridging and how far it
> can be taken (everything via a bridge driver) came from
> discussions with him on SoC ADC handling.
> 
> Lar-Peter Clausen for generally doing insane things with ADCs + having
> one foot in the userspace side of things.

-- 
Regards,

Laurent Pinchart



More information about the Ksummit-discuss mailing list