[Accessibility] Objects contained in documents and tables

Michael Curran mick at kulgan.net
Tue Jan 22 04:19:58 PST 2008

Hi all,

In October last year I raised some issues about objects with in tables 
and documents, in the IAccessible2 subgroup.

The meeting minutes where the issues were talked about can be found at:

As these issues may not just affect IAccessible2, but ATK as well, It 
would be good if these issues were looked at by the wider Open A11Y group.

The general issue is how to best make sure that ATs can provide 
efficient access to documents.

A bit of background

I am the creater and one of the core developers of NVDA (a free and 
open-source screen reader for Windows, mostly written in Python).

One of my beliefs is that if application developers have a wrich but not 
bloated accessibility API, and great documentation to go with it that 
explains best practice covering many situations, this should rappidly 
cut down on the amount of application-specific scripting an AT must 
provide to the user. In general IAccessible2 is a great API, and has 
certainly enabled NVDA wonderful access to such aps as Firefox and Lotus 
Symphony. However, I feel one of the big things missing is generic 
support for documents.

As an AT developer, I would like it if I was able to tell NVDA: "If you 
see a document, just do this"... rather than saying "if this is a 
Mozilla Gecko document do this" or "if this is an IBM Lotus document do 
this"... etc.

In most situations ATs and application developers have decided on some 
general rules such as a list probably only contains list items, and 
there's probably no children inside a button. But when it comes to 
documents, it gets very complex.

I think we can agree on some kind of description of what a document is 
in abstract terms.
*It can be viewed on particular devices or materials as a flat layout
*It has a particular reading order
*It has a caret which can travel the reading order path

As far as an AT is concirned, it usually wants to be able to:
*Navigate the reading order by character/word/line/paragraph
*Do a continuous "say all" which auto navigates by preset chunks along 
the reading order path
*Possibly jump to particular landmarks such as headings, sections, other 
field types
*Posibbly navigate between tabular data such as cells, rows, columns in 
a table
*Detect when moving in and out of particular fields/sections, or detect 
what fields/sections the caret is currently in
*Speak or braille a particular chunk of data to the user, but also 
taking all field and section changes in to account and displaying them 

However, accessibility APIs don't really have a particular protocol for 
communicating with documents.

App developers have devized their own ways of representing documents, 
some of them quite fasinating. However, a lot of them are quite 
inconsistant with each other. And what I would like to see, is some way 
of breaking down these inconsistancies, either by providing much better 
documentation on how to best represent documents, or make sure that the 
accessibility APIs have specific ways of handling documents, so that its 
clear both to the app developer and AT as to how best work with the 

Apps such as Firefox and Lotus Symphony use IAccessible2 to give access 
to documents in such a way so that  the document is a hyerarchy of 
nodes. However, some of these nodes may contain text, and with in that 
text, embedded object characters are used to denote where  a particular 
child of that node should be placed in the reading order of that text.

This is a great solution where the AT is going to traverse the entire 
document and cache their own representation (known usually as a 

However, this way can cause issues when not caching the document, but 
when the user may be right in the middle of it somewhere, and wants to 
know the current line or paragraph etc.

ATs have come up with different logic I'm sure to handle these 
situations, but much of it is probably tied to a specific application, 
and may be less efficient than it could be.

Some ideas I propose we could think about are:

*Provide a way for the AT to ask a particular accessible object if it is 
in a document. Currently the AT must  move up the parent chain untill it 
finds a role of document. Note although most ATs build a chain of 
ancestors when the focus changes, my belief is that in-document logic 
should not only be available at the point of focus, but anywhere, wether 
it be where the AT's navigator object is, or  any type of virtual 
position the AT may provide other than the focus. Perhaps accessible 
objects could have multiple roles, one of them being documentNode, or 
perhaps there can be a documentNode interface that has a way to get to 
the root document node, or perhaps it can be an accessibleRelation that 
can be checked for.

*Like documents, provide a way for an accessible object to be asked if 
it is in a table/row/column, no matter how deep down in the hyerarchy 
with in the table it is. Currently, the AT has to go up the parent chain 
to find the table etc.

*Provide a way for the AT to ask an Accessible object for the next 
object that is in the reading order -- as in where would the caret go 
once it leaves the greater side of this object. Same goes for the other 
way -- previous. Of course the other option is to make sure that the 
hyerarchy reflects this, as in using  depth-first traversal order, it 
happens that you would always land on the next object where the caret 
would be. However this is increasingly becoming impossible as web 
documents are getting way more complex with sections and frames and 
tables etc.

*A way to compare two accessible objects, and find out if 1. they are 
equal, 2. is the first before or after the second in the object 
hyerarchy. For instance,  it may be useful for an AT to represent a 
range in a document, as in two points with in the reading order. 
However, for this to mean something, and so that the AT can handle this 
properly, it must be possible to know  where these two points are in 
regards to each other (lesser, greater, equal).

I think it would be great if we could get the ATs together that 
currently support IAccessible2 / ATK, and really nut out our needs, and 
make sure that the APIs 1. mirror each other, and 2. they provide 
exactly what we really need.

As I am only representing myself here, I am not sure about what other 
ATs' needs really are, or whether these issues even effect them. 
However, I at least think it would be healthy to review all this stuff 
together, and share tips and ideas, and see if we can make things easier 
for ATs and app developers, on more than just one specific platform.

I'm also aware of the big push from certain communities for ATs to try 
and share scripting etc both for apps and the web, but the only way I 
see this ever remotely being a possibility is by making sure that 
IAccessible2 and ATK are improved completely in paralell, and that ATs 
can share ways that they handle particular and generic situations.


More information about the Accessibility mailing list