[Accessibility] Accessibility Minutes 14 June 2006

John Goldthwaite jgoldthwaite at yahoo.com
Thu Jun 22 12:22:25 PDT 2006

6/14/06 FSG Accessibility meeting notes        .
Olaf Schmidt
Gunnar Schmidt                        
John Goldthwaite
Bill Hanaman
George Kraft
Aaron Leventhal
Peter Pricey
Peter Korn
Janina Sajka
Ariel Rios
Cathy Laws
Larry Weiss

Problems with audio recordings for last few calls, hope we have that fixed. [Note: recording didn’t start until page 2 so there are some gaps at the beginning. ]

AT-SPI discussion

a number of RFE’s have been added.  The roles and relations we had consensus on have been added.  Maybe we can focus on some of the others.
George- Collection- current methods, IBM wanted to know about changing match rules.
Larry- ancestor collection, how to enable the use cases for an AT to be able to do.  Confused about term children in the collection interface.  May not want to collect a child of a referenced item.  Want to collect 5-10

Bill- they are children of the collection instance.
Larry - yes, when you want the next 5 or 10.   Reference accessible.  
Janina- Aaron, comments?
Aaron- I’d prefer to have Larry drive the discussion, he understands my issues.
Bill- my current definition is the descent of the collection.  If you think we should use descendent, I’d entertain that.          With recurse flag on.
Larry- from AT perspective, when I’m collecting a subset, with recurse flag on it could be a deep collection            
Bill- designed so that not every object would implement collection.  We envisioned high level objects doing collection.  There could be nested collection but not too deep.  This came out of use cases that were document centric.
Larry- application level, document level or table level for a large table.
Bill- or a form - anything with embedding would have need for embedded collection.
Larry-  what about a browser with document level collection?
Bill- good question, easier to implement if answer is no but puts a burden on the AT.  
Bill- could query collection..
Bill- does anyone have strong feeling about this case?
Peter- it is important to keep subcollections out of the collection
Bill- yes, don’t have to do horrible marshalling
Larry- but for headings or labels or something in other collection. Would it make sense to have a marker - if you hit it.
Bill- it reduces the API calls by one
Larry- other collection in the way
Bill- that is what I was think about when I asked about AT detecting that case, AT knows there a objects that calls to the outermost collector don’t give it.  One more if subsequent queries are going to be incomplete enough.  I’m not sure it is worth inventing a new piece of API to save one call.  Probably should use same behavior and not get more complex.  We do have a match rule.  The client decides on a match rule and then gets a return, It doesn’t get a complex return.  
Hope we don’t have Modify by recipient and return back, that is messy.
Aaron- I’m not sure I agree with that.  If you are asking for a container contents, why wouldn’t you want all contents?
[Start of recording]
Peter- .. There is a lot of glue work we would have to do to make a top level thing that contained something else and imbedding and match that back and forth.  From a use point of view, a lot of AT like a screenreader will be focused on something in particular, like - I’m looking the content region right now, I am wanting to do collections over this content. I think the idea of collections at the application level,  I think there are fewer use cases but maybe I’m not remembering our discussion of them as well.
Aaron- I agree that you would not use them in the application as much but later when we talk about ancestor collection. Mainly in a application you are dealing with focus and point of regard.
Bill- I have a strong view of ancestor collection.
Aaron- don’t want to start discussion on that yet. [Break in recording]

Olaf- European dial-up numbers are not working.

Aaron- collection is mainly for documents but in Gecko you have Frame, Iframe.  Have editable areas.  It is a little confusing, want to operate on a subframe.  Maybe the results that come back should help you determine what it is.  How will you make determination.  When you walk into subtrees, with other 
Bill- my view is that collection is implemented on very specific nodes in the tree.    You have to exercise some care about where you implement collection.  The implementation you face with ATK in Firefox.. Document A inside document B.  Would not want to see a large number of nested documents.  Wouldn’t break on every Iframe.  If you had a wp document with.. Would it make sense to have webpage with several documents inside it.
Larry- optimized collection interface?
Bill- yes to both, anything that implements documents should implement collections.  The bridges look for documents and the should implement accessible on the document.  An ATK document - when an application uses a widget that implements a document, should see that and implement collection.  [Recording resumes] We have a roadmap that calls for ATK collection, the ATK equivalent of the collection interface, to be added to ATK at some time in the future.  We don’t anticipate adding that right away. As soon we do, the application becomes free to add its own optimized implementation of collection and if the Bridge sees ATK collection on the object, it says I don’t need to fake collection, I have one I can use.  That would be the roadmap for applications indicating that they want collection to be implemented and for applications providing their own optimized implementation.
Larry- in the case of Firefox, if it wanted a collection interface on each frame, it would implement expose atk document on each of its frames? 
Bill- yes, that would be the suggestion.
Larry Is that reasonable to you, Aaron?
Aaron- it might be. When I think about this interface, I think about what the W3C DOM does.  It is true if you get element by tag name, it will give you the subtree of elements.  You could be on a group box or you could be anywhere and you can say give me the tag names.  And occasionally it is useful to do that and get all the elements in a subtree. However, it does not walk into subdocuments because it is in a different DOM. That part that is similar but the part  where the person is limited to being to use collection only on a certain things and the starting point can’t be set to anything, to me is, I don’t see a reason to have a limitation. The code is going to be similar either way. Why not open it up and let people use it how they want?
Bill- One reason is because it raises the issue of how deep collection collects.
Aaron- you will have that issue either way, I like the answer- don’t walk into documents. 
Bill- I see, don’t walk into documents as opposed to don’t walk into collections. 
Larry- they would be synonymous at this point.
Bill- yea, right now they would be.
Aaron- What I’m suggesting is allow collection on any element but just don’t walk in a document.
Bill- at the moment collection would only be implemented on documents.
Aaron- yes and I have a problem with that, it’s artificially limiting the usefulness for no particular reason.
Bill- the API was designed as a document centric API.  It is not the same as the W3C DOM, and I think that is okay. 
Aaron- yes but we can learn from the W3C DOM. How can we predict how people are going to want to use it.  Why limit them when we have to write the same code either way?
Bill- limitation is also an advantage, because it helps shape the information and give it some structure
Aaron- what is the advantage?
Bill- having a limit on how deep collection collects. The advantages are two fold, the improved context of the information that we are gathering and greater specificity. That is a mixed blessing. The other is the implementation issue, what is easier to implement depends on the application and the use case.  I agree with Peter, trying to make collection collect completely, that we will collect all the way into the hierarchy is a bad idea.
Aaron- that is not what I’m saying. I’m not saying you should be able to walk into anything. I’m saying you should be able to use anything as a starting point. 
Peter- I don’t understand what you mean by using anything as a starting point.  Say I am on some random element, this element is in a subcollection and that subcollection is in a collection, say an accessible text object in a document in an application. When you say use that as a starting point, a starting point for which collection?
Aaron- It’s descendants. I want to get all its descendants that are role of check box. 
Peter- if the text ..?
Aaron- I’m on a group box in a check box in a dialog. I just want to get all the check boxes in the group box.
Peter- and if the group box doesn’t implement collection then it can’t be done. And you are saying what if group box implements collection or what if the dialog box implements collection?
Aaron- but it is only being implemented on document level objects.
Larry- if the dialog implements collection, then it is not going to be limited to the children of the group box.
Peter- now I have a better understanding of what you are asking about. It is predicated on the assumption that we implementing collection on the dialog box and we can do subtree searches.
Larry- it is more a question of how we do subtree searches.
Peter- it’s more of- is this something we want to support, what is the cost of supporting it and what is the value of supporting it.
Aaron- just from seeing the way things are done in Mozilla, everything is done as a tree. There are trees of documents, trees of elements, trees of layout objects.  So it’s a common to deal with these data structures.  It is the same either way, the starting point and the ending point of the depth first search is the node that you started on so rather than having the requirement that you get everything, limit it to the subtree you want.
Peter- I see some of the usefulness, by putting it in the application space, you saving some on the context switches.  In the dialog box case, I’m not sure how much performance saving. Looking at how things work with GOK right now in similar searches.  Not sure the implementation cost is worth it for what we gain.
Larry- is that true even for trees?
Aaron- image rather than a dialog box, that it is something in a document, say an html document.
Peter- I think it makes tons of sense in a document, that is where this whole API proposal came from.
Aaron- yes but the code is the same either way, it really is.
Peter- it maybe the same for Mozilla, but I don’t see that it is necessarily the same for Swing or Star Office. I’m not convinced that the implementation cost is going to be worth it.
Aaron- I though the infrastructure is where the implementation was going to live in AT/SPI ATK
Bill-it depends, it sounds like you are going to implement it inside the html DOM, you are doing the collecting inside Mozilla
Aaron- we’re not convinced we are going to need to.
Bill- you have to live with collection being implemented on the document only.
Aaron- we’re just discussing it.
Peter- it is an interesting RFE, one that should be considered. I’ll take your word for it that it is the essentially the same code operating on the essentially the same object in Mozilla. I don’t believe that is the case elsewhere. I’m not sure the end user gain is worth the implementation cost. We have so many important things to do, is this going to yield significant end user benefit?  I see no reason we should at the API level, enforce a future in which this is expressly not permitted but I don’t see the value of implementing it.   
Aaron- I hear what you are saying. With the toolkits I’ve worked with there is always a primitive for a kind of an object, in one case it is g object or in Mozilla a DOM element, since we are not saying the subtree collection would contain things from other toolkits, I don’t believe it will cause any developer pain, it is just a more intelligent use of the same coding effort.
Bill- wait, you just said it won’t collect into other toolkits? That is the crux of the problem because..   If we need to bound where collection stops, then we have to have a well defined and easy to ascertain model for when to stop collecting.  Document does not give a good model for that because things that are not documents may choose to implement collection. Tables are a good example, a table is not always a document and it doesn’t always make sense for it to expose the document interface but it may be valuable for table to implement collection.  How else if you say collection does not cross these process boundaries, and we want to Limit depth of collection search that occurs in a mixed context app.
Aaron- I’m not going to be implemented the screenreader but if I were, I would want a consistent model of how collection is implemented.  I would not want some tables to implement collection and others not, that is confusing.
Bill- collection is implemented if you say “do you implement collection?”, and the object says yes it does. We have a interface query based API so we don’t have to say this a table therefore it implements collection.
Aaron- I appreciate that, but on the other hand if you have a special table navigation mode that you are implementing code for.  You know you are in a table, you get the table, if collection is implemented you can do all sorts of tricks but if it is not your hands are tied, you have to do it some other way.
Bill- I understand you don’t want to build too many code paths, but we already have a model for traversing the accessible object DOM. It is a DOM like model of get children, etc. which is implemented on every object.  We have a model that is universal through the hierarchy.  The collections API was not intended to be implemented uniformly by every object but something that was implemented by containers.  
Aaron- but it assumes that the apps vendor has foreknowledge of what will be useful as apposed to the AT vendor deciding what is they need.
Bill- how is that the case?  The app developer needs to be given guideline on where to place the collection boundary. Where to tag something as a collection boundary or where to tag..
Aaron- I’d like to hear from AT developers, whether they think collection should be implemented for every object or if they are fine with toolkits having collection on various objects.
Peter- Speaking as a former AT developer, there is a clear distinction between content and chrome.  The challenges of access to chrome of one nature and the challenges of access to content are of a different nature. The AT developer regularly uses two fairly distinct models, you can’t have it deal with the two problem. With the chrome problem, all objects that are there are instantiated on the screen and are in memory, have objects behind them and are easily found and manipulated. With content, the object could be a 2k file or 2 Megabyte file.  There is no way you are going to have objects instantiated for everything that is there.  We had kind of a hack for this manages descendants and collection is what grew out of accessible documents. But you need to have a better way for looking at and enumerating things that don’t have individual objects instantiated in memory that we can go find and query. 
Aaron-so it is easier to implement collection on a dialog than on some document?
Peter- might be easier to implement collection on a dialog but is not be needed in the way that is needed for content because there no other efficient way to get everything in a document because
Aaron- how do you get the default button in a dialog without traversing every single thing?
Peter- you don’t without traversing everything.  The challenge of traversing everything when you have a 50 items isn’t that much of an issue. Whereas in the 2MB file it is impossible. That is why I was saying I’m not convinced the performance optimization you get from implementing collection in dialog boxes gives us a benefit that is worth the implementation costs. I see the performance optimization it yields but I’m not sure that with modern processor speeds, that we need to do this.    

Bill- the notion of implementing this in the bridge means that the burden of doing it is relieved to some extent. We may not have to implement a dialog specific collection, once we have built the generic machinery in ATK Bridge, it should work for most sorts of objects.  We will only need the ATK collection application specific implementation for the hard cases that were the initial motivation for the API.  I don’t see disadvantages and I see little cost for implementing it on dialogs but I don’t see the value of implementing it on everything. A number of problems there, I don’t see a value, it doesn’t matter how easy it is to do.
Peter- I see the point Aaron was making, if we are doing this in the Bridge, it is the same code so  why not do it everywhere?
Bill- no, If we are doing it in the Bridge, the Bridge has to know where to do it. By doing it on every object, we undermine the value of the interface by implementing it on every object.
Peter- this ties back to the use case, you can see it making sense on top level items, on dialogs, on windows.   What is the current ancestor of this window that has focus, what is set of check boxes in the toolbar?  You can see questions like that making sense. I see Aaron’s point that if you are implementing it in the Bridge, it is kind of the same code.  I also see the problem of dealing with embedded objects, like Gecko or Bonobo embedding.  I also see the problem of scope, if I am in the toolbar, what are focused objects or what are the list of buttons.  Do I want to get the buttons in the form that is not scrolled onto the screen?
Larry- how do we limit the scope of that collection if it is only in that top level frame?
Bill- that is what collection is designed to do. It is designed to let you scope what gets returned.  It lets you scope for visibility, for a number things.
Aaron- but not for a subtree.
Larry- and not for the same parent.
Bill- another fly in the ointment in collecting at a lower level: in GUI’s it is not unusual for the tree structure to not map nicely on the visual presentation. It is possible you may think you are in the containing group box but you are not. You could miss things that the layout engine has put next to the group that visually and conceptually are associated with it.  The client would be vulnerable if it thought the hierarchy is semantically meaningful all the way down.
Larry- if it is not meaningful how is the AT to do things even without collection?
Peter- you don’t make that assumption. The AT doesn’t make that offer of presentation, the AT says here are to buttons in the dialog box, not here are the buttons in the grouping because group may not be defined.  If no grouping is defined in is not going to attempt to present one.
Bill-you are never going to be immune from this problem entirely. There are going to be situations where the AT would like to use those container relationships.  It would be nicer if we could semantically tag the nodes, the containers in the hierarchy that are most meaningful. In a way implementing collection on specific nodes is a way of saying this node is an important one.  
Larry- I thought you were saying the application could do that by saying this implements document?
Bill- that is the current way, right now we would only get collection for- let’s just say the Bridge has to know where to do it. Right now, the only way to know where to do it was in association with other ATK interfaces, for instance ATK document. Maybe ATK Table. Implementing collection in the Bridge for all tables would be a good idea. Right now there is no way other than the document for the application to flag it. But it is early days, the road map calls for applications to implement their own flavors of collection and doing it in an optimized way. The application is free to provide a collection interface on any node it wishes, that is what the roadmap calls for.  As Eric says we don’t want to do it until we know we need it.

Peter- Sorry to have to leave the call but I wanted to thank Aaron and Peter for explaining this issue. 
Larry- you mentioned the roadmap as far as things coming in the future, is that public somewhere.  
Bill- the roadmap is what we’ve been building in these meetings. 
Larry- I was just wondering if there was a web document that I just hadn’t found yet. 
Janina- sounds like we could use a document.
Bill - we have a email trail and several documents. Maybe we need a definitive document.  it is important to talk about boundary conditions.

Larry- Let me summarize what we said so far on boundary conditions- a collection does not traverse into another collection. We expect the Bridge to implement collection on the top level frame or application if for no other reason to allow collection of collection interfaces.  As far as Collect children is concerned, you are referring to the children of elements that implement the collection interface. So when you say next, you are not limiting to the subtree that is referenced you are still collecting from that point down to the set of children that are the children of the original collection. It took me a while to come up with that one. One of the next children also refers to itself as next match, which is not a bad choice of terms but it may be thought of that.  
Bill- that is not in the IDL is it?
Larry- yes, If you read next children it references itself as next match. At least it did the last time I looked and it was relatively recent.
Bill- I’m not sure about the implementation of collection on top level applications. Let’s say we leave that one as an open question. Collections not collecting instances of itself,  I’m hearing we have a majority view but also a minority view from Aaron.  If a collection didn’t collect inside of itself, it would make Aaron’s idea a non-runner, since it couldn’t recurse.
Aaron- I agree but it should be possible to limit what you are walking inside of.
Bill- we all agree that boundary conditions are important for collection and appropriate. There should be non-trivial boundary conditions and to stop when it hits certain boundaries.
Aaron- what about to start at a certain boundary?
Bill- yes, it needs to start somewhere as well. We have a difference of opinion on how often those boundaries, how condition should appear.
Aaron- like use this subtree but don’t use subtrees like this. You could say something like  look for state defaults on buttons in this dialog box.
Bill- no, that’s a new idea.  When I am talking about boundary conditions, I mean boundary conditions that are non-client specified and are characteristic of the way collection works. In other words, the boundary conditions are imposed on clients and clients should be aware of them, as opposed to the client gets to specify the boundary conditions.   Client gets to specify a lot of things but it doesn’t get to specify all the boundary conditions.  That is a trade off on burden in the  implementing code and in the client.

Larry- Continuing with the summary, only one more thing, document would be the marker interface the application could use to say indicate collection here.
Bill- We are agreed that document is a marker interface that can be used.  Whether we need more that one is an open question.  I think we could wait a while on that.  We agree that collection is implemented on document, if we need it more and we want it more broadly, we can rethink it.  Right now there are two ways for collection to be implemented, one is for ATK document to appear and Bridge implements collection on behalf of that and the other is for ATK collection itself to be implemented by the application. The second isn’t in CVS yet because the plan is that we didn’t need it right away.
Larry- we were going to see if would be need and if it would be requested by any clients.
Bill- saying document is where collection is implemented isn’t the final word but it maybe the right place for us to stop our short term Gnome 2.16 discussions.
Larry- I’m okay with that, I was just summarizing.  I one more question, is there anyway to start at the bottom and collect backward?
Bill- I’m not sure what you mean by collect backward.  In collection there are a couple of sense flags, what do you mean by bottom?
Larry- if I were to specify a match rule such that I would get three things, a loose match rule.   something on accessible.
Bill- if you asked for it in reverse order, you get them backward.  The sort rules allow you to get things backward.
Larry- if I said backward but give me only one it would give me the last one?
Bill- that’s right, it doesn’t say give me the first one, it says give me one and I want them in reverse order.  If you say give me 10 in reverse order, counting backward it will give you the last 10.

Larry- I thought the answer was yes, I just wasn’t sure how. Is there are a reason we are specifying  Getchildren as well as Getnext and Getprevious, Instead of having a null accessible to indicate the top?
Bill- the answer is to avoid the null. Since we have getnext and getprevious, it doesn’t make sense for them to do the same thing if you pass a null object in. That seems weirder than having the three API calls. Either we should have 3 or collapse it to one and have that be one of the parameters to collecting.  I’ll look back at the API to see if there are any differences.
Larry- I though the only difference was the one parameter.  If I do getprevious and give me 10, and I don’t say reverse order, what order are they in?  Are they the most recent, closest to the current?
Bill- It is a bounds question, it will start walking at the top and it will stop walking at the one
Larry- starting from the top, it will give me the last ten before the one I specified.
Bill- only if you say reverse order
Larry- reverse order?  
Bill- lets assume the maximum number of items returned will be larger than the size of the set. Hopefully, a common case, unless we have enormous collections. Most clients will set that fairly big and that a lot of the time they won’t fill the set they ask for.  In that case Getprevious gives you everything up to a point, it just indicates where the collection process starts. Whereas Getnext indicates where the collection process begins.  Getprevious plus Getnext will give you the same as Getchildren. The sort order is the same for all 3 APIs, they are not reversed in sense.  The sort order is always from the nominal beginning of the search to the end of the search, it is just that getprevious tells the search where to stop and Getnext tells the search where to begin.

Larry- say I am in the middle of a very large document and I say Getprevious and set count to ten, does it give me the first 10 in the document collection?
Bill- yes, it will give you 10 items that match this criteria and stop collection when you reach this node but it will stop sooner because it will fill the bucket.
Larry- humm...
Bill- I think I need to change the document because the phrase “relative to a subelement” does not really tell you what is going on.  The subelement is the boundary condition, the search stops when it gets to the subelement. That is a Documentation bug.
Larry- Getprevious would seem to mean start here and work backward.
Bill- that needs to be fixed
Larry- can you update the documentation to clarify it.  
Bill- I’ll do that.  I promised to create a branch, may be I did before went on sabbatical and forgot all about it.  Good point.
Larry- we are looking at how to implement this that was why we requested the discussion.  We’re out of time but we still wanted to have a discussion about ancestor.

Bill- My thought is that ancestor doesn’t fell like a fit to collection.  An ancestor is always a kind of straight vector up.  Semantically it doesn’t feel like the same sort of activity.  Second, if we put it anywhere that might to something that was reasonable to get for any object.  Don’t see a reason for it to obey boundary conditions
Larry- to play Devil’s advocate, why is GetActiveDescentents in collections?  Isn’t it the reverse of what you are talking about.  
Bill- you mean in container don’t you?
Larry- no, when you look at the collection IDL one of the messages is GetActiveDescendent.
Bill- I’ll have to look at that.        

Janina- we are well past time, we can continue this on either of the next to calls.  We are also trying to put some audio discussions together in June.
Bill- GetActiveDescendent is interesting, we may not need it at all, it looks like our collection rules are broad enough that we should be able to construct it.  It doesn’t  return a vector, just a single object.
Larry- we can’t specify that we want it to be a child of a current object, a specific object. 
Bill- there can only be one active descendent
Larry- But I might want to know the active descendent is not a child of this one. 
Bill- I remember what the issues around this is but we are over time.  This is more about tables, big complicated objects. 
Larry- lets defer that until next time. Maybe we can describe some of the scenarios on the list.

John Goldthwaite
jgoldthwaite at yahoo.com
828 885-5304
Sneak preview the  all-new Yahoo.com. It's not radically different. Just radically better. 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.linux-foundation.org/pipermail/accessibility/attachments/20060622/792cc17c/attachment.htm

More information about the Accessibility mailing list