[Accessibility] Objects contained in documents and tables
mick at kulgan.net
Tue Jan 22 04:19:58 PST 2008
In October last year I raised some issues about objects with in tables
and documents, in the IAccessible2 subgroup.
The meeting minutes where the issues were talked about can be found at:
As these issues may not just affect IAccessible2, but ATK as well, It
would be good if these issues were looked at by the wider Open A11Y group.
The general issue is how to best make sure that ATs can provide
efficient access to documents.
A bit of background
I am the creater and one of the core developers of NVDA (a free and
open-source screen reader for Windows, mostly written in Python).
One of my beliefs is that if application developers have a wrich but not
bloated accessibility API, and great documentation to go with it that
explains best practice covering many situations, this should rappidly
cut down on the amount of application-specific scripting an AT must
provide to the user. In general IAccessible2 is a great API, and has
certainly enabled NVDA wonderful access to such aps as Firefox and Lotus
Symphony. However, I feel one of the big things missing is generic
support for documents.
As an AT developer, I would like it if I was able to tell NVDA: "If you
see a document, just do this"... rather than saying "if this is a
Mozilla Gecko document do this" or "if this is an IBM Lotus document do
In most situations ATs and application developers have decided on some
general rules such as a list probably only contains list items, and
there's probably no children inside a button. But when it comes to
documents, it gets very complex.
I think we can agree on some kind of description of what a document is
in abstract terms.
*It can be viewed on particular devices or materials as a flat layout
*It has a particular reading order
*It has a caret which can travel the reading order path
As far as an AT is concirned, it usually wants to be able to:
*Navigate the reading order by character/word/line/paragraph
*Do a continuous "say all" which auto navigates by preset chunks along
the reading order path
*Possibly jump to particular landmarks such as headings, sections, other
*Posibbly navigate between tabular data such as cells, rows, columns in
*Detect when moving in and out of particular fields/sections, or detect
what fields/sections the caret is currently in
*Speak or braille a particular chunk of data to the user, but also
taking all field and section changes in to account and displaying them
However, accessibility APIs don't really have a particular protocol for
communicating with documents.
App developers have devized their own ways of representing documents,
some of them quite fasinating. However, a lot of them are quite
inconsistant with each other. And what I would like to see, is some way
of breaking down these inconsistancies, either by providing much better
documentation on how to best represent documents, or make sure that the
accessibility APIs have specific ways of handling documents, so that its
clear both to the app developer and AT as to how best work with the
Apps such as Firefox and Lotus Symphony use IAccessible2 to give access
to documents in such a way so that the document is a hyerarchy of
nodes. However, some of these nodes may contain text, and with in that
text, embedded object characters are used to denote where a particular
child of that node should be placed in the reading order of that text.
This is a great solution where the AT is going to traverse the entire
document and cache their own representation (known usually as a
However, this way can cause issues when not caching the document, but
when the user may be right in the middle of it somewhere, and wants to
know the current line or paragraph etc.
ATs have come up with different logic I'm sure to handle these
situations, but much of it is probably tied to a specific application,
and may be less efficient than it could be.
Some ideas I propose we could think about are:
*Provide a way for the AT to ask a particular accessible object if it is
in a document. Currently the AT must move up the parent chain untill it
finds a role of document. Note although most ATs build a chain of
ancestors when the focus changes, my belief is that in-document logic
should not only be available at the point of focus, but anywhere, wether
it be where the AT's navigator object is, or any type of virtual
position the AT may provide other than the focus. Perhaps accessible
objects could have multiple roles, one of them being documentNode, or
perhaps there can be a documentNode interface that has a way to get to
the root document node, or perhaps it can be an accessibleRelation that
can be checked for.
*Like documents, provide a way for an accessible object to be asked if
it is in a table/row/column, no matter how deep down in the hyerarchy
with in the table it is. Currently, the AT has to go up the parent chain
to find the table etc.
*Provide a way for the AT to ask an Accessible object for the next
object that is in the reading order -- as in where would the caret go
once it leaves the greater side of this object. Same goes for the other
way -- previous. Of course the other option is to make sure that the
hyerarchy reflects this, as in using depth-first traversal order, it
happens that you would always land on the next object where the caret
would be. However this is increasingly becoming impossible as web
documents are getting way more complex with sections and frames and
*A way to compare two accessible objects, and find out if 1. they are
equal, 2. is the first before or after the second in the object
hyerarchy. For instance, it may be useful for an AT to represent a
range in a document, as in two points with in the reading order.
However, for this to mean something, and so that the AT can handle this
properly, it must be possible to know where these two points are in
regards to each other (lesser, greater, equal).
I think it would be great if we could get the ATs together that
currently support IAccessible2 / ATK, and really nut out our needs, and
make sure that the APIs 1. mirror each other, and 2. they provide
exactly what we really need.
As I am only representing myself here, I am not sure about what other
ATs' needs really are, or whether these issues even effect them.
However, I at least think it would be healthy to review all this stuff
together, and share tips and ideas, and see if we can make things easier
for ATs and app developers, on more than just one specific platform.
I'm also aware of the big push from certain communities for ATs to try
and share scripting etc both for apps and the web, but the only way I
see this ever remotely being a possibility is by making sure that
IAccessible2 and ATK are improved completely in paralell, and that ATs
can share ways that they handle particular and generic situations.
More information about the Accessibility