[Accessibility] Minutes of 8/30 DBUS AT-SPI discussion

[John Goldthwaite]

I'm still working on transcription of the meeting notes but here is the first half of the DBUS AT-SPI discussion.

8/30/06 DBUS AT-SPI  meeting notes        .
Attending:        
            

Harrald Fernengel
Gunnar Schmidt
Olaf Schmidt
Bill Hanaman
Janina Sajka
John Goldthwaite
Gary Cramblett
Peter Farrenti
Larry Weiss
George Kraft
    

Approach for supporting AT-SPI in dbus.
Janina- what is the time frame for accessibility in KDE?
Olaf- at the moment we are developing the module. QT4.2 will be out soon and we do have the accessibility module in there.  We are brushing it up for Mac and Windows and dbus bridge should benefit from that as currently it is standalone.
Janina- is this in the QT for small devices?
Harrald- we have 1/3 share Linux market for Qtopia, and we have released phone ourself completely open source, entire stack is open source so you can play with it. At the moment we don’t have accessibility on it. We have a push from management to get it in so I will get the resources. 
Janina- this meeting is to discuss supporting AT-SPI in dbus.
Harrald- it started in the last KDE meeting in which we sketched out kde4. A big question was how to handle the accessibility; we discussed various options- supporting atk, moving AT-SPI to dbus. Since we are moving to almost completely to dbus, the old interprocess communications will be dropped. New technologies are already using dbus like the multimedia architecture, the removable device architecture, we made a prototype.  I tried 5 or 6 different approaches and came up with something promising.  Started with the most used AT-SPI interfaces– we now have accessible object which relays information on what children and which parents are rectangle.  Accessible text which lets you read text and accessible application which is the root application which you can query for the focus and will inform you about focus changes.  You can get the entire object tree, the rectangle, the position of every object. We have the SPI value in place so you can set the values and move it around.
  The API is pretty much inspired by AT-SPI.
Bill- we are familiar AT-SPI architecture what would be more interesting would be more about the dbus approach and how you get your bindings, and more about dbus. Not everyone on the call is familiar with dbus.
Harrald- dbus is a freedesktop.org project to get an IPC on the free desktop that can be used by gnome and kde. KDE has dcop for IPC introduced in version 2. Introduced in kde2 for application interprocess communication.  This technology had the disadvantage since it is based on qt so not used by Gnome. Dbus was agreed upon as the IPC that can be used by both desktops. It is completely self sustained, dbus is just linking to C so it is very portable.  It has been ported to Microsoft, Mac OS10 and all the unixes. It is currently shipped with all linux distributions, used by the linux kernel to pass information about hardware changes, the hardware abstraction layer.   It is extremely easy to set up, just compile, install and run. The program model is that they don’t have their own deep nesting like CORBA. They hook into the available toolkits. 
Bill- could you explain what you mean by deep nesting?
Harrald- They don’t have their own network code for socket notify, they don’t have their own event loop.  They integrate with the current toolkit- python, gtk, qt and so on. They  use native possibilities which keeps the base library small.
Bill- that is not specified by CORBA, it is just that orbit calls the gmain loop.  There is nothing in CORBA that says that an orb has to implement its own.
Harrald- that is correct. Unless you write your own orb implementation, you are stuck with merging two different event loops which leads to all sorts of problems.  Introduce new event loop ? Have their own Event loop and handling for locking and so on already.
Bill- I agree that if you link to orbit, the nature of orbit is that you are pulling in a main loop and if you have your own main loop there is some redundancy. It is not clear that main loop integration issues are intractable. There should be some prospects for main loop integration.  I know there are some projects.  There have been some challenges for main loop integration but they have been solved in other projects- open office had their own main loop worries.  Merging main loops occurs in other environments.
Harrald- I tried for about 3 weeks to integrate with gloop and gave up. I did not get enough information, the other way was to run everything in gloop and let gloop be the event loop. This produced a lot of other porblems so it is not an option at present.
Bill- I understand that this a big problem and needs help. But  I don’t think that is really a dead end.  There should be a solution but just not an obvious one and we need some help.
Harrald- if you give me some code to convince me.  The only attempts at integration between qt and gloop and they all made gloop into the designated event loop. I haven’t seen any other way around yet. If you could point me to some code, I’d be super happy.
Bill- I think there is a solution in the open office code.  I think the solution they are using does not call the gloop main loop, instead they pull events out of gloop one at a time.
Harrald- but that means polling
Bill- I don’t think you have to poll. You can do it by selecting, they get a signal when there is something to ready to read.
Harrald- that is exactly the path that I was trying but I wasn’t getting the socket notifiers that gloop communicates on so I don’t know what to select on.  My hack was just to probe every so often to see what sockets were open.
Bill- did you ask the glib core guys about this?
Harrald-we had some communication but their answer was why don’t you run qt and glib.  For the imbedding thing that would add a lot of stuff.
Janina- action item- somebody needs to provide some code examples of main loop integration.
Harrald- yes an example of how to do it with timers and socket notification.
Bill- we can ping Michael Meeks about that.  He said that he though that it was possible and he was the person who did the work in open office so he should know where the code is.

Harrald- we evaluated the technologies, we had a sample implementation to see how we can move AT-SPI to dbus.  We have the entire wiget set accessible.  All we need to do is add more of the properties and functionality that AT-SPI gives us.  We did not yet agree on a solution on how to get the AT-SPI registry but I see that as more of a technical detail and shouldn’t be a show blocker.  I mentioned the cross platform on dbus which is very important for us. We have imbedded technologies, we have our own phone out and we need to move on that platform where memory is very constrained.  Dependency is only on C so very portable. Language bindings - we have qt, we have Glib bindings, mono bindings, Python Bindings.  You can access the entire functionality from the shell.  Can query from the shell to get objects. You can query for the text, type or state.  You don’t need any high level programming to do simple scripting or to invoke a button.  The code expansion or code size for
 the entire thing is quite light weight. We have XML descriptions of the interfaces, create headers and cbp, all you have to do is fill out a bit of code.  Don’t have to worry about the usually problems- where do I get my environment, do I have to reference count, who owns the memory. If I pass a string it is always a question, who owns it, what is the encoding.  In dbus you don’t have all these problems because the integration with toolkits is seamless. You just use the toolkits native string class. For instance, we have a rectangle in there that can be translated into any rectangle class that any toolkit offers.  A lot of the complexity vanished over night.  In a week we got the same functionality as 3 months work on atk integration. So it is a very painless way to get it implemented.
Bill- but the implementation is just a subset of the text API.  
Harrald- yes there is more to come.
Bill- the business about native rectangles, I’m not sure what the advantage is. I’m not sure I see the importance of that. The IDL defines structures so you need to send structures over the wire that are toolkit independent.  If you do something in a structure that is native to glib, that means any implementation of atk has to understand those glib primitives.  If you push a qt primitive, then that binding has to understand qt.
Harrald- not necessarily- you can marshall it into a dbus structure so when I push over a rectangle I send 4 integers and everyone can de-marshall that because the de-marshaller can handle that. If I push a string, it is a string, you don’t have any translation in there.
Bill- We have that in the IDL since that is part of the architecture. Are you saying that it was easier to write to that than to atk?
Harrald- It is so easy because you use your native tools and push that stuff all in the library. If I have a structure with 4 integers and I marshall it as a rect. In your code you see a rect rather than an artificial structure where you have to do the translation yourself.
Bill- but there is no rect defined in some languages.
Harrald- then you define your own, define your own structure and create a custom marshaller in the binding layer. 
Bill- I don’t understand what you are trying to do.  I don’t understand your assertion about a native structure. What do you mean by a native rect?
Harrald- the structure that a programmer uses in his toolkit to represent a rect. Every gui toolkit has something like a rect. The nice thing is you get this over the bus, the marshaller, you use your tools- 
Bill- you are saying the dbus has toolkit specific marshallers?
Harrald- yes that’s right.   Implementation, can ask for text, text until end of paragraoh, etc.
the object hierarchy- in CORBA have to reference, de reference.  In dbase you have an object path, don’t have to worry about
Bill- how do you keep an object alive?
Harrald
Bill- you can’t force a wiget to stay alive but you can force an accessible object to stay alive.
Harrald- object stuff- I’ll have to see a case.  I don’t see that as a disadvantage.  Code interface, code reads much nicer and is less error prone.  Have to check if interface is valid.
Bill- reference count
Harrald- we see and advantage in not asking developer to write code.
Bill- not what we have been discussing
Harrald - is it part of the standard?
Bill- yes, disadvantages in reference counts
Harrald- dbus
Bill- will be invoking the AT-SPI standard.  There is no proposal to write directly to dbus.  
Harrald- if you have a wrapper around it, why not?
Bill- rep and un-rep counts performed over the wire.
Harrald
Bill- 
Harrald- why should I ref something if isn’t required by dbus.
Janina- is this
Gary- what Harrald is saying is that dbus is not a component technology, n Bill is saying that AT-SPI needs to worry about object lifetime.  Depends upon it.  I don’t see a need to worry about object lifetimes for most things.  When the widget goes away, the accessble object goes away.  If orca tries to access it, it gets an error.

Bill- C based based on g-objects, necessary because C has no concept of garbage collection.
Gary - something that AT clients create to see if coming from an accessible object.  That one worries be, might be better if AT-SPI returned a state sequence.
Bill- want objects to be obaque- I’d like to talk to you about that off line.  I’d like to go to a different life cycle tracking method but .. Don’t dissagree with Harrald long term.  Certain create as result of a query.  Will exist until object says I’m done with it.  Thinks like states F which are tiers of wigets will have to be adapted.
Gary- I thought of doing reference counting over dbus but not sure
Bill- the pattern for AT-SPI is lots of very small messages.  The ref counts would not be as big component as you think.  Could you explain how the XML and dbus bindings are created?

Harrald- the XML is hand created, have xml to cdp?  XML and generate interface from them.
Bill- what are the prospects of an automated way of creating the XML?  Gary..
Gary- 
Harrald- IDL defines methods and arguements.  If we are moving away from CORBA, why keep the CORBa
Bill- we need to keep CORBA

Bill- looking at your code, which piece would be the wrapper that I would be calling?  Accessible H?
Harrald- skeleton, adaptors.  Just publish the adaptors.  XML to CBT gives you the interface that you would then call.  Pretty much an object with those methods as outlined in the XML/ If it runs the interface, you would that back.
Bill- so these would just be strings, just a way of identifying the object
Harrald- from the string you can see the hierarchy and go up.
Bill- that is different.
Harrald
Bill- as a side effect of the implementation you get a whole chain of unique strings?
Harrald- yes
Bill- how large does the data get?
Harrald- ask for it relative to the parent so you don’t have transfer the whole thing.  Don’t know about nesting in gui.
Bill- there could be hundreds, especially a page with html- If you do ask for the reference, since it is an opaque, doesn’t have to pass the parent in to get the origional reference.
Harrald= have to concatenate
Bill- you would have to concatenate a lot, the fact that dbus doesn’t have concrete object reference, worries me
Harrald
Bill- Gary you have a different approach?
Gary- I have to work with what ever Harrald gives me.
Bill- ours object reference are opaque.  Need to think what you would do if you have to deal with a highly nested structure.  random number..
Harrald- browser with deep nesting, use   something. I will put this down as an action item for me.
Bill- looked at the code, my question are how your interfaces are created from the IDL and how this could be bridged with the existing technologies.
Harrald- exchanging with Gnome project.
Bill- I’m not sold on using XML for specifying this type of interface.  See what looks like a lot of hand coded ... how much of a reduction in hand coding could you achieve?
Harrald- AT-SPI wants strings, didn’t hand write the API, could ask give adaptor.

Janina- need to see how much more of this type of discussion we need?  We have the sample code as an action item.
Bill- Accessible object .cct was that output by automation or hand coded.
Harrald- that was hand coded but I didn’t have to.  Check the accessible
Accessible object was all hand coded, the others are automated.
Bill- it is almost cut and paste.  Gary- how does your work relate to Harralds.
Gary- I want what I am doing to work with what Harrald is doing.  I agree.
Bill- still a bit fuzzy how it works together.
Gary- I start with iddl, a compiler,  
Bill- this is exciting and interesting work but we have some serious talking to do about how to make this interoperate.  The ideal case would be for everyone to be using a dbus wire protocol.  I would be concerned about any departure from what comes out of the IDL.  We don’t want an incompatible implementation.  Want some that is binary compatible with dbus.
Harrald-
Bill-
Harrald- free standards group, compatible
Bill- if you find that some areas where you can’t use the IDL compiler, let us know so we can fix it.  We want to be able modify the IDL and have it generated, not do any hand coding.
Olaf- if you use IDL to XML compile, you could use the output of that for the dbus version. We could use that compiler to generate the XML every time there is a change in AT-SPI.



John Goldthwaite
jgoldthwaite at yahoo.com
828 885-5304
 		
---------------------------------
Do you Yahoo!?
 Get on board. You're invited to try the new Yahoo! Mail.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.linux-foundation.org/pipermail/accessibility/attachments/20060906/fd929f39/attachment.htm